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Abstract 

Past  research  proposed  that  it  is  possible  to  forecast  cargo  demand  using  time  series 
models  and  that  there  exists  potential  cost  savings  in  the  way  that  Civilian  Reserve  Air  Fleet 
(CRAF)  is  used  for  cargo  airlift.  United  States  Transportation  Command  (USTRANSCOM) 
perfonns  annual  "fixed-buys"  of  CRAF  to  support  airlift  needs.  Forecasted  cargo  demand  would 
allow  for  reasonably  accurate  cargo  projections  vs.  the  current  expected  value  estimation. 
Accurate  forecasting  allows  for  greater  "fixed-buys,"  further  incentivizing  CRAF  airlines  as  well 
as  reducing  the  number  of  additional  aircraft  purchases  during  the  quarterly  and  monthly  buys. 
Multiple  forecasting  models  are  constructed  and  the  results  compared.  A  Monte  Carlo 
simulation  using  a  discrete  pallet  destinations  distribution  and  a  discrete  pallet  port  arrival  date 
distribution  (based  on  historical  data)  outputs  a  month  of  projected  pallet  weights  (with  date  and 
destination)  that  are  equivalent  to  the  forecasted  cargo  amount.  The  simulated  pallets  are  then 
used  in  a  heuristic  cargo  loading  algorithm.  The  loading  algorithm  places  cargo  onto  available 
aircraft  (based  on  real  schedules)  given  the  date  and  the  destination  and  outputs  statistics  based 
on  the  aircraft  ton  and  pallet  utilization  as  well  as  number  of  aircraft  types  used  and  the  total  cost 
of  the  projected  airlift  schedule.  A  technical  approach  to  the  operational  planning  of  cargo  airlift 
could  provide  significant  cost  savings  or  could  provide  an  alternative  planning  approach 
changing  the  future  of  USTRANSCOM  operations. 
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OPERATIONAL  PLANNING  OL  CHANNEL  AIRLILT  MISSIONS  USING 

LORECASTED  DEMAND 


I.  Introduction 


Background 

The  United  States  Transportation  Command  (USTRANSCOM)  is  located  at  Scott  Air 
Force  Base,  IL  and  is  the  single  manager  of  the  U.S.  global  defense  transportation  system  (DTS). 
USTRANSCOM  is  composed  of  three  component  commands,  the  Army’s  Military  Surface 
Deployment  and  Distribution  Command,  the  Navy’s  Military  Sealift  Command,  and  the  Air 
Force’s  Air  Mobility  Command  (AMC)  (USTRANSCOM  History,  2012).  AMC  is  also  located 
at  Scott  AFB  and  provides  aerial  refueling,  medical  evacuation,  passenger  transportation,  and 
cargo  delivery  anywhere  around  the  world.  AMC’s  passenger  transportation  and  cargo  delivery 
capabilities  are  supported  by  the  Civil  Reserve  Air  Fleet  (CRAF)  program  which  allows  the 
Department  of  Defense  (DOD)  to  contract  aircraft  from  civilian  airlines  to  meet  national 
objectives.  One  of  the  more  important  aspects  of  the  CRAF  program  is  its  ability  to  provide 
additional  commercial  airlift  aircraft  in  times  of  national  emergencies. 

Within  AMC  is  the  618th  Tanker  Airlift  Control  Center  (TACC)  which  is  responsible  for 
planning,  scheduling,  and  directing  more  than  1,300  mobility  aircraft  for  a  number  of  operations 
to  include  strategic  airlift.  The  618th  TACC  includes  the  Global  Channel  Operations  Directorate 
(XOG)  which  manages  worldwide  strategic  airlift  operations  and  employs  military  aircraft  such 
as  the  C-5  Galaxy  and  the  C-17  Globemaster  III,  along  with  civilian  aircraft  to  fulfill  airlift 
requirements  (618th  Tanker  Airlift  Control  Center,  2008). 

AMC  and  TACC  are  responsible  for  organizing  U.S.  tanker,  airlift,  and  inter-theater 
aeromedical  evacuation  capabilities  worldwide.  Airlift  during  the  Global  War  on  Terror 
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(GWOT)  has  significantly  evolved  since  Desert  Stonn,  causing  a  direct  increase  in  the  demand 
on  the  available  air  assets.  Airlift  historically  supports  forces  from  behind  rather  than  acting  as  a 
frontline  operation  (Second  Line  of  Defense,  2011).  However,  with  altered  deployments,  airlift 
has  received  much  more  forward  pressure  than  ever  before.  Within  in  the  last  ten  years,  a  heavy 
commitment  to  Central  Command  to  airlift  all  of  the  necessary  equipment,  machinery, 
weaponry,  and  rations  to  the  deployed  troops  has  become  the  norm.  Unfortunately,  there  are  not 
enough  organic  aircraft  available  to  airlift  everything  that  is  needed.  A  crucial  element  of  TACC 
has  become  its  ability  to  leverage  commercial  assets  or  CRAF.  CRAF  is  utilized  for  channel 
missions  or  regularly  scheduled  flights  that  fly  fixed  routes  on  a  predetermined  schedule.  The 
CRAF  schedule  is  generated  from  USTRANSCOM  airlift  requirements  based  on  historical  data. 
As  of  now,  there  is  no  mathematical  projection  of  the  airlift  needs  for  the  future  (Second  Line  of 
Defense,  2011).  The  number  of  organic  aircraft  that  are  used  for  airlift  is  dependent  on  the 
number  that  are  not  dedicated  to  other  missions  and  that  are  in  good  condition  to  fly.  These 
organic  aircraft  are  then  used  for  either  military  missions  or  to  carry  outsized  cargo  that  cannot 
fit  in  a  civilian  aircraft.  TACC  then  augments  the  organic  forces  with  the  civilian  contract 
charters.  As  of  201 1,  80%  of  airlift  needs  were  met  by  commercial  carriers  (Second  Line  of 
Defense,  2011). 

Initially  born  during  World  War  II,  CRAF  was  officially  created  by  executive  order  in 
1951.  The  creation  allowed  for  the  nation  to  have  a  contingency  plan  to  meet  airlift  requirements 
in  times  of  need  (emergencies  or  wartime)  when  airlift  exceeded  the  capabilities  of  military 
aircraft  fleet  (Knight  &  Bolkcom,  2008).  The  current  relationship  between  the  military  and  the 
civilian  airline  industry  was  made  official  in  1987  by  President  Ronald  Reagan’s  National  Airlift 
Policy.  The  policy  stated  that  in  order  to  protect  national  security  interests  in  a  wartime 
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environment,  the  military  and  the  civilian  airlift  would  be  interdependent  in  meeting  wartime 
airlift  requirements.  An  important  feature  of  the  CRAF  program  is  that  it  is  still  voluntary  for 
airlines  to  participate.  Fortunately  for  the  U.S.  military,  the  incentive  program  is  strong  enough 
to  maintain  a  relationship  with  32  airlines  (Michael  W.  Grismer,  2011). 

Although  CRAF  has  proven  itself  invaluable  to  the  military’s  airlift  needs,  the  fact 
remains  that  the  program  is  very  expensive  (Michael  W.  Grismer,  2011).  The  expense  is 
necessary  for  the  most  part,  but  with  the  current  economic  crisis  and  the  need  to  cut  costs,  the 
CRAF  program  presents  an  area  where  large  savings  are  feasible.  The  key  to  achieving  cost 
savings  is  a  more  technical  approach  to  the  planning  that  already  occurs  when  purchasing 
civilian  aircraft  for  our  future  airlift  needs. 

To  bring  the  cost  of  CRAF  into  perspective,  since  September  11,  2001  (9/1 1)  the 
Department  of  Defense  (DoD)  exceeds  $3  billion  per  year  in  CRAF  costs  when  the  DOD  only 
allocates  $2.5  billion  each  year  for  the  CRAF  program  (House  of  Representative  Hearing,  111 
Congress,  2009).  While  this  allocation  is  an  enormous  amount  of  money,  the  cost  for  the 
military  to  build  up  and  maintain  the  same  number  and  capacity  of  aircraft  over  the  life  of  the 
program  in  inflation-adjusted  dollars  is  possibly  $128  billion  (House  of  Representative  Hearing, 
111  Congress,  2009).  A  new  747-8  costs  approximately  $180  million  for  one  plane  and  while 
we  reserve  many  different  sized  aircraft,  we  still  request  approximately  1100  civilian  aircraft  a 
year  (House  of  Representative  Hearing,  1 1 1  Congress,  2009).  It  is  quite  easy  to  imagine  how  the 
costs  add  up  if  the  DOD  were  to  replicate  the  necessary  inventory  number  as  organic  military 
aircraft  and  provide  all  of  the  additional  support  necessary. 

There  do  not  seem  to  be  any  documents  that  firmly  state  what  the  cost  of  a  particular  type 
of  civilian  aircraft  is  to  fly  for  the  CRAF  program.  For  this  study,  previous  estimates  from  past 
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studies  will  be  used  (Lindstrom,  2012).  The  Boeing  747  is  one  of  the  most  common  CRAF 
aircraft,  and  the  estimated  cost  of  contracted  airlift  from  Dover  AFB  to  Bagram,  Afghanistan  is 
$460,000.  Let  us  say  that  there  are  two  747s  contracted  to  fly  on  the  same  day  to  the  same 
location,  but  there  is  only  enough  cargo  to  fill  one  plane.  The  options  are  to  fly  both  planes 
regardless  if  one  is  full  or  not,  or  to  fly  one  and  cancel  the  second.  Many  people  would  probably 
think  if  a  plane  is  already  purchased,  then  why  would  one  want  to  cancel  it?  This  is  the  school  of 
thought  that  TACC  has  adopted  and  applies  in  their  methodology.  The  issue  is  it  is  cheaper  to 
cancel  an  aircraft  than  to  fly  it.  To  think  of  the  cancellation  cost  as  a  scheduling  fee  provides  a 
different  perspective  to  the  example.  The  DOD  has  paid  $100,000  per  747  to  secure  the  aircraft 
to  fly  a  military  mission  on  that  day  of  the  month  and  will  pay  an  additional  $360,000  to  actually 
fly  the  aircraft  on  the  scheduled  day.  The  DOD  will  pay  $200,000  total,  but  will  only  pay  the 
additional  cost  $720,000  if  both  aircraft  are  flown.  If  one  flight  is  cancelled,  then  both 
reservations  fees  are  still  paid,  but  there  is  only  one  $360,000  payment  ($100,000  +  $100,000  + 
$360,000  =  $560,000).  The  $560,000  is  significantly  less  than  the  full  $920,000. 

A  new  planning  methodology  suggesting  that  flights  be  cancelled  if  a  minimum  cargo 
weight  is  not  met  would  contribute  to  cost  savings.  Often  times,  the  main  goal  is  to  shorten  port 
hold  times,  or  the  time  until  cargo  departs  the  base.  Cancelling  flights  may  cause  some  port  hold 
times  to  increase  since  the  cargo  would  wait  until  enough  cargo  arrived  for  the  plane  to  meet  the 
minimum  weight  requirement.  This  study  will  show  that  the  port  hold  times  will  not  be 
significantly  affected  and  cargo  will  still  depart  in  a  timely  manner.  The  identification  of  the 
issue  with  cancellations  and  the  possible  savings  involved  led  to  a  number  of  studies,  in  turn 
providing  the  foundation  for  this  thesis.  This  led  to  pointing  out  the  main  problem,  finding  a 
methodology  to  predict  the  amount  of  cargo  that  will  be  freighted.  The  annual  buy  is  based  on 
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the  predicted  amount  of  freight  tonnage.  If  this  amount  can  accurately  be  projected  for  a  full 
year,  it  may  resolve  over-scheduling  or  under-scheduling  contracts.  Since  intennediate  buys  are 
then  completed  quarterly  and  monthly,  the  unknown  surges  of  cargo  will  also  be  included  in 
short  term  forecasts. 

Problem  Statement 

Cargo  demand  changes  frequently  and  rapidly  in  a  wartime  environment.  It  is  very 
difficult  to  specify  twelve  months  in  advance  the  number  of  aircraft  one  would  need  to  airlift  a 
hypothetical  number  of  cargo  tons.  Cargo  demand  is  dependent  on  a  number  of  factors  such  as 
the  timeframe  of  the  war,  the  set  up  of  the  forward  operating  bases,  and  whether  or  not  U.S. 
troops  are  surging.  Current  demand  is  the  expected  value  from  previous  years  (Second  Line  of 
Defense,  2011).  If  planners  are  able  to  forecast  demand  then  it  is  also  possible  to  simulate  and 
optimize  the  number  of  commercial  and  organic  assets  needed  to  lift  the  cargo  and  provide  a 
rough  schedule  as  well. 

An  additional  problem  is  the  cost  of  flying  unnecessary  aircraft.  The  cost  of  cancelling  a 
CRAF  is  significantly  less  than  the  cost  of  flying  the  aircraft,  meaning  it  is  possible  to  save 
money  just  by  cancelling  the  flight.  If  cancellations  are  viewed  as  sunk  reservation  costs, 
unrelated  to  the  additional  cost  of  flying,  then  it  is  easier  to  envision  cost  savings  by  flying  the 
least  amount  of  aircraft  as  possible.  Most  savings  are  still  generated  by  the  benefit  of  accurate 
predictions  and  only  scheduling  what  is  actually  needed. 

Objectives 

The  primary  objective  is  to  examine  the  effect  of  advanced  mathematical  modeling  on  the 
operational  planning  and  scheduling  of  CRAF  aircraft.  The  effects,  on  both  cost  and  schedule,  of 
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optimizing  the  number  of  the  aircraft  available  will  be  quantified  and  analyzed  in  a  number  of 
scenarios  providing  a  detailed  look  at  the  solution. 

Hypothesis 

The  hypothesis  of  this  research  is  two-fold.  First,  current  estimates  of  cargo  demand  for 
future  years  demand  are  only  moderately  effective  and  an  advanced  mathematical  model  would 
provide  more  accurate  upfront  purchase  numbers  and  less  underutilized  CRAF  purchases. 
Second,  a  simulation  that  optimizes  and  provides  an  overall  average  for  the  allocation  of  the 
cargo  demand  and  the  optimal  airframe  needs  can  realize  further  gains  in  the  accuracy  of  the 
purchase  and  ensure  that  airlift  is  being  used  to  the  DOD’s  and  military’s  advantage. 

Assumptions 

There  are  not  many  assumptions  necessary  in  moving  forward  with  solutions  for  this 
problem.  The  problem  is  handled  in  a  way  to  provide  an  analysis  of  alternatives  to  the  decision 
maker  and  to  include  sensitivity  analysis  as  well  in  order  to  cover  as  many  uncertainties  as 
possible  in  the  problem.  The  main  assumption  to  be  addressed  comes  when  using  Boots  on  the 
Ground  (BOG)  as  an  input  in  the  forecasting  models.  All  forecasts  are  based  off  of  98,000 
troops  being  present  each  month  in  Afghanistan  for  all  of  FY 1 1 .  This  estimate  is  provided  by 
congressional  estimates  for  the  Global  War  on  Terror  (Belasco,  2011). 

The  simulation  uses  peacetime  aircraft  load  planning  factors  to  replicate  real  world 
planning.  CRAF  is  currently  inactive  and  planning  occurs  at  peacetime  requirements  so  the 
allowable  cabin  loads  (ACLs)  are  set  to  a  lower  value  (Mobility,  2012).  The  aircraft  are  capable 
of  carrying  greater  capacities,  but  those  allowances  are  to  be  considered  in  wartime  environment. 
Figure  1  provides  a  quick  look  at  the  assumed  ACLs  and  number  of  pallet  positions  available  on 
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the  organic  and  civilian  transport  aircraft.  These  parameters  are  the  basis  of  the  simulation  and 
any  comparison  statistics  that  occur  later  on. 


Aircraft 

Type 

Pallet 

Positions 

Cargo  (Stons) 

Passengers46 

Standard  NEO 
Passengers 

ACC1 

Planning' 

ACL 

Planning 

C-130 

6 

17 

12 

90 

80 

92/74’ 

C-130J 

8 

22 

18 

145 

128 

128 

C-17 

18 

65 

45 

101 

90 

101 

C-5 

36 

89 

61 

73 

51 

73 

K.C-10 

(Airlift) 

23 

60 

32 

75 

68 

75 

K.C-135 

(Airlift) 

6 

18 

13 

53 

46 

53 

A-330 

- 

- 

- 

- 

240 

266 

B-747 

33 

120 

104 

2% 

296 

347 

B-757 

13 

38 

33 

115 

115 

175 
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Figure  1  Aircraft  Payload  During  Peacetime  Planning  (Air  Mobility  Planning  Factor,  2011) 


Limitations 

This  research  is  limited  to  the  specified  Area  of  Responsibility  (AOR)  of  Afghanistan 
since  U.S.  forces  have  been  withdrawn  from  Iraq.  Further  limiting  the  analysis  is  the  focus  on 
Dover  AFB,  DE.  There  are  other  ports  that  operate  in  the  same  nature,  but  the  large  majority  of 
cargo  shipments  to  the  AOR  depart  from  Dover  AFB  due  to  its  location  on  the  East  Cost  as  well 
as  its  primary  mission  being  large  scale  airlift. 

There  are  limitations  in  any  forecast  due  to  the  type  of  information  used  and  the  type  of 
problem.  With  this  problem,  only  monthly  data  is  being  considered  due  to  having  limited 
information  on  additional  inputs.  The  monthly  data  is  only  available  starting  in  2005  through 
2011,  which  limits  the  potential  data  points  to  81  observations.  12  of  these  points  are  withheld 
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for  validation  further  reducing  the  series  to  69  observations.  This  is  a  feasible  set  of  data  points, 
but  it  may  have  been  easier  to  work  with  a  larger  data  set. 

The  last  limitation  is  the  original  data  set,  or  the  Global  Air  Transportation  Execution 
System  (GATES)  data  that  is  used  for  the  problem.  It  is  possible  that  the  parsing  process  to  be 
described  in  Chapter  3  does  not  include  all  available  infonnation.  This  is  due  to  limited 
knowledge  and  experience  with  aircraft  mission  codes  and  pallet  IDs. 

Implications 

Planning  ahead  is  always  beneficial,  but  in  this  case,  planning  ahead  can  reduce  the  cost 
of  the  CRAF  program  to  the  DOD.  This  study  will  show  that  there  are  more  accurate  ways  to 
predict  cargo  demand  that  are  still  easily  implemented.  The  benefit  of  accurate  predictions  at 
each  purchase  stage  is  tremendous.  There  are  abundant  cost  savings  by  scheduling  aircraft  to  fit 
the  actual  demand  versus  scheduling  aircraft  based  on  historical  averages.  The  analysis  in 
Chapter  4  will  show  that  it  is  possible  to  reduce  the  number  of  planes  flown  in  a  month  and 
better  utilize  the  planes  provided  to  provide  significant  cost  savings  when  compared  to  planning 
methods  in  the  past. 
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II.  Literature  Review 


This  chapter  discusses  the  usefulness  of  CRAF  so  the  reader  can  gain  an  understanding  of 
what  the  program  provides  and  why  airlift  schedules  currently  exist  as  they  are.  There  is  also  a 
discussion  of  forecasting  techniques  and  how  they  are  selected  and  forecasting  with  air  cargo 
demand.  Using  forms  of  regression  and  time  series  modeling  is  not  foreign  to  the  Department  of 
Defense  (DOD)  or  the  United  States  Air  Force  (USAF).  There  have  been  multiple  research 
papers  written  regarding  military  demand  and  airlift  forecasting  in  a  wartime  environment,  but 
there  do  not  seem  to  be  any  papers  or  articles  presented  in  popular  literature  about  the  combined 
topics.  The  literature  review  presented  below,  provides  a  brief  overview  of  the  main  topics  and 
the  combined  application  will  be  presented  in  later  chapters. 

CRAF  Program 

As  highlighted  earlier,  in  order  to  meet  wartime  needs,  the  DOD  supplements  its  organic 
military  aircraft  with  commercial  carriers  through  the  CRAF  program.  The  great  benefit  of  the 
CRAF  program  and  one  of  the  main  reasons  why  it  is  still  so  prevalent  in  today’s  wartime 
environment  is  that  it  minimizes  the  cost  to  the  government  by  guaranteeing  aircraft  availability 
during  a  surge  without  the  added  cost  of  maintaining  organic  capabilities.  A  general 
understanding  of  the  program  and  its  benefits  and  drawbacks  may  be  necessary  to  derive  further 
value  from  this  study.  The  DOD  spends  billions  of  dollars  annually  to  make  substantial  use  of 
participating  carriers.  Participating  carriers  must  make  their  aircraft  and  aircrews  available  on 
short  notice,  with  some  of  the  suddenness  mitigated  through  two  different  types  of  business 
purchases  known  as  a  “fixed  buy”  and  an  “expansion  buy.”  The  “fixed  buy”  requires  that  the 
DOD  submit/prepay  for  a  percentage  of  their  airlift  services  upfront  for  the  coming  year,  thus 
signing  a  one-year  contract.  All  additional  requirements  or  additional  airlift  services  are 
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purchased  at  multiple  points  within  the  year  during  an  “expansion  buy.”  The  “fixed  buy”  is 
guaranteed  payment  a  year  in  advance  and  therefore  was  a  strong  incentive  for  CRAF  partners  to 
continue  participating  in  the  program.  The  benefit  of  the  “fixed  buy”  for  the  DOD  is  to  ensure 
that  routine  missions  to  transport  people  and  cargo  to  overseas  stations  are  already  accounted  for 
(Arthur,  2007). 

Since  the  beginning  of  the  wars  in  Iraq  and  Afghanistan,  the  fixed  buy  has  averaged  about 
20  percent  of  CRAF  business.  It  was  foreseen  that  “expansion  buys”  would  increase  due  to 
deployments  closer  to  areas  of  operation.  What  was  not  predicted  were  the  large  increases  in  the 
cargo  being  airlifted  overseas  and  how  the  DOD  could  have  decreased  the  “expansion  buys”  by 
looking  forward.  In  2007,  it  was  proposed  that  the  DOD  provide  more  guaranteed  buys  based  on 
expected  requirements  (Arthur,  2007). 

A  common  question  that  has  arisen  throughout  the  years  is  “Why  doesn’t  the  military 
purchase  their  own  aircraft?”  The  CRAF  program  is  by  no  means  free;  the  DOD  pays  rates 
based  on  weighted  average  carrier  costs.  This  minimal  cost  was  approximately  $1.5  billion  a 
year  during  Desert  Stonn  and  is  much  greater  now.  The  cost  to  the  Air  Force  to  acquire  and 
support  the  number  of  additional  aircraft  adequate  for  a  major  crisis,  including  aircrews,  and 
maintenance,  at  that  time,  would  have  been  anywhere  from  $15  to  $50  billion.  It  could  be 
argued  that  the  DOD  has  or  will  soon  pay  the  same  amount  out  of  pocket  already,  but  the  costs  to 
the  Air  Force  would  constantly  grow  due  to  continued  support  of  the  aircraft,  as  well  as  upgrade 
costs  or  aircraft  replacement  costs(Knight  &  Bolkcom,  2008).  Under  that  realization,  the  use  of 
CRAF  by  the  DOD  is  easily  justified,  and  the  focus  has  been  on  how  to  revamp  the  incentives  to 
the  carrier  participants  to  keep  them  involved  in  the  program. 
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In  current  CRAF  operations,  excesses  in  the  CRAF  capacity  numbers  have  been  identified  as 
a  possible  risk  by  the  Government  Accountability  Office’s  evaluation  of  the  DOD’s  Mobility 
Capabilities  and  Requirement  Study  2016  (Borseth,  et  al.,  2010).  By  reviewing  and  analyzing 
past  schedules  and  the  amount  of  cargo  airlifted,  this  claim  is  easily  verified  and  was  identified 
as  a  portion  of  the  problem  statement  in  Chapter  1.  The  CRAF  program  is  critical  to  the  DOD’s 
cargo  airlift  capability  as  well  as  its  ability  to  meet  our  national  objectives  (Solis,  et  ah,  2009).  It 
is  important  to  better  manage  such  an  important  capability  effectively  to  reduce  wasteful 
spending  on  behalf  of  the  government  as  well  as  to  reduce  the  cost  implications  on  the  airlines 
themselves  for  near  term  scheduling. 

Forecasting  Techniques 

There  are  numerous  regression  and  forecasting  techniques  available  to  apply  to  nearly 
any  scenario  with  data.  Below  are  descriptions  of  the  techniques  that  are  applicable  to  this  study. 

Simple  linear  regression  assumes  that  there  is  a  relationship  between  the  dependent 
variable  (Cargo)  and  the  independent  variable  (BOG)  that  can  be  approximated  by  a  straight  line. 
Quadratic  and  polynomial  regression  is  a  form  of  nonlinear  modeling  that  incorporates  the 
equation  of  a  simple  parabola  or  a  more  complex  curve  with  the  coefficients  or  slopes  of  the 
independent  variables  representing  the  rate  of  curvature  of  the  shape  or  the  trend.  Time  series 
regression  is  a  form  of  polynomial  regression  that  uses  time  as  the  independent  variable 
(Bowerman,  O'Connell,  &  Koehler,  2005). 

Smoothing  models  include  simple  moving  average,  exponential  smoothing,  and  Holt- 
Winter’s  Method.  Exponential  smoothing  is  a  method  that  accounts  for  changing  trends  and 
seasonal  factors  over  time.  More  recent  observations  are  given  more  weight  compared  to  older 
observations.  Exponential  smoothing  has  additional  variations  of  the  broader  technique  to 
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account  for  strong  trends  in  seasonality,  such  as  Brown’s  Method  or  Damped  Trend  Method. 

For  this  research  Holt-Winter’s  Method  (Additive),  Seasonal  Exponential  Smoothing,  and 
Simple  Exponential  Smoothing  were  the  three  best  smoothing  models.  Smoothing  models 
applied  to  cargo  forecasting  was  completed  in  recent  research  efforts,  allowing  for  this  research 
to  focus  more  advanced  techniques  (DeYoung,  2012). 

Box-Jenkins  Models,  more  commonly  known  as  Auto-Regressive  Integrated  Moving 
Average  (ARIMA)  models,  are  forecasting  models  that  are  used  to  describe  stationary  time  series 
in  terms  of  relationship  between  data  and  the  forecast  errors.  A  time  series  is  stationary  if  the 
mean  and  the  variance  of  the  series  are  constant  through  time.  If  not,  the  series  is  considered  to 
be  nonstationary  and  must  be  transfonned.  An  autoregressive  (AR)  model  refers  to  a  model  that 
expresses  the  current  time  series  value  as  a  function  of  past  time  series  values.  A  moving 
average  (MA)  model  refers  to  a  model  that  uses  past  random  shock  values  (errors)  to  predict  the 
current  time  series.  A  random  shock  value  is  the  difference  between  the  actual  value  and  the 
forecasted  value  better  known  as  the  residual  and  it  describes  the  effect  of  any  remaining  factors 
outside  of  the  time  series  on  the  model  (Bowennan,  O'Connell,  &  Koehler,  2005).  An  AR  and 
an  MA  model  can  be  combined  to  create  a  mixed  model  where  all  components  are  represented. 
The  “I,”  or  integrated,  component  represents  whether  or  not  a  difference  transformation  is  used 
on  the  data  series.  A  first  difference  of  the  time  series  values  is  equivalent  to  the  current  time 
series  value  at  time  t  minus  the  previous  time  series  value  at  time  t-1.  This  transformation  is 
applied  to  the  entire  data  series  to  create  a  new  working  series  (Bowennan,  O'Connell,  & 
Koehler,  2005).  Differencing  transformations  can  be  applied  to  both  the  nonseasonal  and  the 
seasonal  components  of  the  data  series  to  create  an  updated  “working”  series. 
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The  ARIMA  model  is  represented  in  the  fonnat  ARIMA(/;,t/,t/),  where p  represent  the 
order  the  AR  term,  d  represents  the  order  the  differencing  term  and  q  represent  the  order  of  the 
MA  tenn.  A  time  series  that  includes  seasonal  factors  can  be  modeled  in  a  more  advanced  form 
to  include  additional  ARIMA  terms  purely  related  to  the  seasonal  occurrence  of  the  data  (every 
12  months  for  example). 

An  advanced  ARIMA  model  allows  for  an  independent  time  series  to  be  used  to  predict  a 
dependent  time  series.  There  are  two  popular  methods  to  perform  this,  the  first  is  simpler  and  is 
known  as  an  intervention  model.  Intervention  models  are  normally  used  when  some  kind  of 
extreme  event  occurs  such  as  a  natural  disaster  that  may  impact  the  values  of  the  data  set  such  as 
a  production  capacity  (Bowennan,  O'Connell,  &  Koehler,  2005).  In  this  case,  there  is  no 
extreme  event,  but  there  is  still  an  external  event,  BOG,  that  may  or  may  not  affect  the  variable 
to  be  forecast,  or  cargo.  The  result  is  to  estimate  a  linear  regression  model  that  describes  the 
intervention  (BOG)  and  then,  while  assuming  the  error  terms  are  statistically  independent, 
develop  an  ARIMA  model  that  adequately  describes  the  error  terms.  The  ARIMA  model  of  the 
error  terms  can  be  substituted  for  the  error  tenn  at  the  end  of  a  general  regression  model. 

The  second  form  is  more  advanced  and  is  known  as  a  transfer  function  model  and 
predicts  future  values  of  a  time  series  on  the  basis  of  past  values  of  one  or  more  related  time 
series.  It  is  similar  to  a  regression  model,  but  now  includes  a  serially  dependent  response,  inputs, 
and  error  tenns.  The  same  modeling  method  applied  to  single  time  series  ARIMA  models  is 
applied  to  the  transfer  function  method,  but  is  an  iterative  process  used  on  multiple  time  series. 
Combining  forecasts  to  improve  prediction  performance  is  also  applied  to  multiple  transfer 
function  model  as  well  as  to  a  transfer  function  and  other  single  time  series  ARIMA  models. 
(Bowennan,  O'Connell,  &  Koehler,  2005)  The  combined  forecasts  method  allows  for  linear 
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combination  of  the  two  forecasts  from  at  least  two  different  methods  to  be  created.  The  forecasts 


of  two  methods  can  be  combined  to  create  a  single  superior  forecast  than  either  model  alone. 
(Montgomery,  Jennings,  &  Kulahci,  2008).  The  three  forms  of  ARIMA  modeling  described 
above,  standard  ARIMA,  intervention  models  and  transfer  functions  are  all  modeling  forms 
applied  and  compared  with  the  exponential  smoothing  methods  in  this  research.  The  comparison 
and  analysis  of  the  techniques  begins  in  Chapter  3. 

ARIMA  model  building  is  a  three-step  iterative  procedure  that  first  requires  a  tentative 
model  to  be  formed  through  analysis  of  the  historical  data.  Secondly,  the  unknown  parameters 
are  estimated  and  lastly,  residual  analysis  detennines  model  adequacy  and  room  for 
improvements  (Montgomery,  Jennings,  &  Kulahci,  2008).  This  brief  outline  of  the  procedure 
may  make  the  process  seem  quick  and  insignificant,  which  at  times  may  be  the  case,  however,  a 
much  more  detailed  discussion  on  all  three  steps  is  required  to  actually  apply  the  iterative 
technique  correctly.  Only  the  steps  of  the  procedures  that  are  directly  applicable  to  the  model 
development  will  be  quickly  explained  below. 

Estimating  an  ARIMA  model  requires  an  initial  evaluation  of  the  raw  data  to  determine  if 
the  data  set  is  stationary  or  has  constant  variance.  As  seen  in  the  left  hand  plot  (Figure  2)  of  the 
cargo  data  there  is  nonconstant  variance  displayed  by  the  large  differences  between  the  increases 
and  decreases  in  the  data  points  and  by  the  increasing  mean,  both  hinting  that  a  first-difference  is 
necessary.  A  first-difference  transformation  is  applied  and  the  data  set  (Figure  3)  is  significantly 
altered  and  now  stationary. 
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Figure  2  JMP  Raw  Data  Plot  of  Cargo  Figure  3  JMP  Plot  of  Transformed  Cargo  Data 


Once  the  data  is  stationary,  the  Auto  Correlation  Functions  (ACF)  and  Partial  Auto 
Correlation  Functions  (PACF)  are  reviewed  for  lags  of  significance  and  to  detennine  the  manner 
in  which  the  consecutive  lags  decrease.  These  indicators  describe  the  parameters  and  the  type  of 
model  (ex.  AR  or  MA)  that  best  fits  the  data.  Significance  in  the  lags  represents  the  presence  of 
autocorrelation  between  the  time  series  data  points.  Looking  at  the  ACF,  the  left  hand  function 
in  Figure  4  below,  a  significant  spike  at  the  first  lag  means  that  we  are  rejecting  the  null 
hypothesis  that  the  autocorrelation  is  equal  to  zero.  The  function  also  displays  a  dying  down 
trend.  The  PACF,  the  right  hand  function  in  Figure  4,  shows  one  near  significant  spike  and  a 
slow  dying  down  trend.  Dying  down  on  both  the  ACF  and  the  PACF  represents  a  mixed 
parameter  model  with  the  number  of  parameters  corresponding  to  the  number  of  significant 
spikes  in  the  ACF  and  the  PACF. 
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Figure  4  JMP  Output  of  the  ACF  and  PACF  for  the  Cargo  Demand 


The  ACF/PACF  analysis  above  describes  the  following  ARIMA  forecasting  model 
ARIMA^  1,1,1) .  There  is  one  AR  term,  one  differencing  transformation,  and  one  MA  term. 

There  is  also  no  seasonality.  Figure  5  shows  the  data  transformation  completed  by  subtracting 
the  past  term  from  the  current  term  to  create  a  new  working  series.  Figure  6  shows  the 

theoretical  form  of  the  ARIMAy  1,1,1)  model.  The  estimated  series  (Figure  7)  is  the  fully 
constructed  model  with  the  actual  estimated  parameters. 
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Yr  =  Actual  Values 
Yt  =  Forecasted  Values 
ZT  =  Transformed  Values 

T  =  Time/Period 
8  =  Constant  derived  from  Mean 
(f>  =  Autoregressive  Coefficient 
6  =  Moving  Average  Coefficient 
aT  =  Random  Shock 
B  =  Backshift  Operator 

Table  1  Definition  of  ARIMA  Parameters 


Working  Series  ZT=S+  q iZr_,  +  aT  - 0xaT_{ 

ZT  =  (1  -  B)  Yt=Yt- Yt_x  yt  -8+  (j){YT_x  -  YT2  ^  +  aT-0l  [yt_x  -  YT_X  j  +  YT_X 

Figure  5  First  Difference  Transformation  Figure  6  Theoretical  ARIMA  (1,1,1)  Model 

Yt  =  23.87  +  (0.47)(yr_1  -  YT_2  )  +  aT  +  (0.76) (yt_,  -  YT_X )  +  YT_X 

Figure  7  Estimated  ARIMA(1,1,1)  Model  for  Cargo  (tons) 

When  estimating  a  model,  diagnostic  procedures  are  followed  to  ensure  that  the  model  is 
adequate.  This  is  done  through  residual  analysis.  If  the  appropriate  AR  and  MA  orders  have 
been  identified,  the  residuals  of  the  observations  are  transfonned  to  white-noise  processes.  The 
residual  ACF  and  PACF  are  reviewed  to  detennine  if  there  is  any  significant  structure  left  in  the 
data  that  was  not  identified.  If  there  is  no  significant  difference  between  the  lags,  the  residuals 
indicate  the  current  form  of  the  model  is  good.  The  Ljung-Box  statistics  for  each  lag  detennines 
whether  or  not  there  is  a  relationship  between  the  residuals,  if  there  is  not,  then  the  value  is  small 
indicating  that  the  model  is  adequate.  We  see  below  in  Figure  7  that  there  are  no  significant  lags 
as  well  as  no  significant  Ljung-Box  values  and  the  associated  p-values  are  also  large  indicating 
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the  same.  Additionally,  there  are  near  significant  terms  at  the  7th  lag  on  both  charts.  It  would  be 
worth  experimenting  with  a  seasonal  model  to  reduce  the  autocorrelation  further. 
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Figure  8  RACF  and  RPACF  for  ARIMA(1,1,1)  Cargo  Model 


Once  the  model  is  developed  and  determined  to  be  adequate,  the  model  development 
process  is  complete.  The  next  step  is  to  determine  whether  or  not  that  model  in  particular  is  the 
best  one  for  the  data  set  at  hand.  A  variety  of  goodness  of  fit  checks  can  be  used  for  the  model 
selection  and  comparison.  A  brief  description  accompanies  the  more  important  criteria  below. 

The  mean  squared  error  (MSE)  measures  the  variability  in  the  forecast  errors  with  the 
goal  of  small  variability  in  the  forecast  errors.  The  MSE  is  a  direct  measure  of  the  variability  of 
a  one-step  ahead  forecast  error  (Montgomery,  Jennings,  &  Kulahci,  2008). 
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(1) 


n  t= 1 


A  maximally  valued  R 1  statistic  is  equivalent  to  minimizing  the  sum  of  the  squared 
residuals  (Montgomery,  Jennings,  &  Kulahci,  2008).  Large  values  of  the  statistic  represent  a 
good  fit  of  the  predicted  values  to  the  historical  data.  One  has  to  be  careful  in  using  this  value  as 
it  can  be  easily  inflated  by  adding  insignificant  parameters  to  the  model.  Having  a  parsimonious 
model  is  usually  preferred  when  dealing  with  regression  models  or  time  series  models. 


R2  =1— 


H(y<-yY 


The  Akaike  Infonnation  Criteria  (AIC)  and  the  Schwarz  Infonnation  Criteria  (SIC)  are 
both  similar  criteria  that  penalize  the  sum  of  squared  residuals  for  including  insignificant 
parameters  in  the  model  (Montgomery,  Jennings,  &  Kulahci,  2008).  For  both  criteria,  a  smaller 
value  is  desired.  As  you  can  see  below,  the  penalty  for  the  AIC  is  smaller  than  that  of  the  SIC. 
When  these  criteria  are  implemented  in  statistical  programs,  slight  variations  in  the  calculation 
may  exist.  For  example,  the  equations  below  are  traditional  textbook  equations  and  the  program 
JMP  (JMP  10,  2012)  implements  a  slightly  different  variation  of  both  criteria  where  the  penalties 
are  slightly  different. 


I*,2 

AIC  =  In  — — 
T 


SIC  =  In 


pHt) 


The  Mean  Absolute  Percentage  Error  (MAPE)  and  the  Mean  Absolute  Error  (MAE,  also 

known  as  Mean  Absolute  Deviation  or  MAD)  are  shown  below.  The  MAPE  is  meaningful  in 
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that  it  represents  a  scaled  size  of  the  forecast  error  in  the  model  by  being  displayed  as  a 
percentage.  The  MAE  is  useful  depending  on  the  unit  of  measurement  of  the  dataset 
(Montgomery,  Jennings,  &  Kulahci,  2008).  In  this  case,  knowing  the  relative  size  of  the  cargo 
and  using  the  same  data  with  different  models  allows  us  to  use  both  criteria  for  model  selection. 


MAPE  =  -Y}\ret{\)\ 
n  t= 1 


(5)  MAE  =  -Y\et(\) 

n  i=\ 


(6) 


An  important  aspect  of  model  evaluation  is  to  remember  that  none  of  these  criteria  should 
be  used  individually  to  select  a  best  model.  Certain  criteria,  the  MSE,  R2 ,  and  the  AIC,  do  not 
penalize  the  criterion  value  enough  for  including  extraneous  parameters,  making  them 
inconsistent.  The  SIC,  includes  a  “heavier”  penalty  for  this,  allowing  for  consistency  and 
parsimonious  models. 


The  estimated  coefficients,  as  seen  in  the  Figure  10,  are  all  determined  to  be  significant. 


The  goodness  of  fit  criteria  are  displayed  in  Figure  9  below. 
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Schwarz’s  Bayesian  Criterion 
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65  Stable  Yes 
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0.81897104 
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Term 
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Std  Error 
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Estimate 

AR1 

1 

0.474403 

0.27242 

1.74 

0.0863 

23.8693125 

MAI 

1 

0.764807 

0.20308 

3.77 

0.0004* 

Intercept 

0 

45.413685 

31.26421 

1.45 

0.1512 

Figure  10  ARIMA(1,1,1)  Parameter  Estimates 


Figure  9  ARIMA(1,1,1)  Model  Summary 

Developing  a  transfer  function-noise  model  is  a  more  complex  process  with  additional 


steps  to  determine  the  model  parameters.  It  may  be  necessary  to  make  stationary  both  the  input 


data  and  the  output  data  in  different  manners.  The  next  step  is  to  prewhiten  the  data. 
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Prewhitening  involves  first  finding  an  adequate  ARIMA  model  that  describes  the  input  series 
(Bowerman,  O'Connell,  &  Koehler,  2005).  The  next  step  is  somewhat  more  complicated.  Most 
statistical  programs,  in  this  case  JMP,  have  the  ability  to  prewhiten  data.  It  uses  the  parameter 
estimates  from  the  estimated  ARIMA  model  on  the  input  series  and  solves  for  prewhitened 
output  values.  The  goal  of  which  is  to  obtain  a  sample  cross-correlation  function  (SCC)  with  a 
filtered  input  and  a  filtered  output  that  can  be  used  to  identify  the  parameters  of  the  transfer 
function  model.  The  SCC  is  a  measure  of  the  linear  relationship  between  the  prewhitened  input 
and  output  values. 

The  s  operator  measures  the  number  of  past  transformed  input  values  influence  on  the 
transfonned  output  values.  The  r  operator  measures  the  number  of  its  own  past  values  that  the 
output  is  related  to.  These  operators  are  observed  at  both  the  nonseasonal  and  the  seasonal  level 
just  like  in  standard  ARIMA  model  development.  These  appropriate  operators  can  now  be  input 
into  the  general  transfer  function  model  to  determine  if  all  of  the  parameters  are  in  fact 
significant  to  the  model  or  not.  At  this  point,  none  of  the  parameters  for  the  output  series  have 
yet  been  estimated.  The  residual  ACF  and  PACF  charts  must  be  analyzed  once  a  parsimonious 
input  model  has  been  found.  The  residual  ACF  and  PACF  charts  identify  an  appropriate 
ARIMA  model  to  describe  the  error  terms,  or  the  residual  cargo  data  unaccounted  for  by  the 
input  model.  The  final  transfer  function  model  is  thereby  obtained  by  the  error  term  ARIMA 
model  and  the  transfer  function  inputs.  The  same  model  adequacy  process  must  be  continued  at 
this  point  to  determine  if  all  parameters  are  in  fact  significant,  if  the  residuals  are  not  significant, 
and  to  detennine  if  enough  correlation  is  accounted  for  by  the  model.  Continuing  this  process 
resulted  in  two  significant  transfer  function  models  to  compare  with  the  other  models  presented 
above. 
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Forecasting  Demand 

For  many  years  now,  forecasting  has  been  taken  into  consideration  for  operational 
planning  in  the  civilian  world.  Manufacturers  (ex.  car  companies),  airline  companies,  city 
planners,  hospitals,  and  many  other  areas,  businesses,  or  institutions  incorporate  mathematical  or 
statistical  methods  in  their  strategic  planning  for  future  operations.  Managers  or  executives  have 
long  been  asked  to  make  a  decision  about  the  future  and  can  potentially  benefit  by  applying  the 
general  principles  of  forecasting.  One  of  the  first  decisions  to  make  is  what  type  of  forecasting  is 
necessary  for  the  problem  at  hand.  There  are  two  types  to  consider,  qualitative  or  quantitative, 
and  the  selection  depends  on  the  context  of  the  forecast,  availability  of  data,  the  accuracy  needed, 
the  period  to  forecast,  and  the  cost/benefit  of  the  forecast  (Chamber,  Mullick,  &  Smith,  1971). 

By  weighing  these  factors,  a  decision  maker  can  choose  an  appropriate  method  or  technique  for 
the  problem  at  hand. 

Qualitative  forecasting  is  used  when  data  are  not  available;  therefore,  logical  judgment  is 
used  to  estimate  factors.  Quantitative  forecasting  incorporates  both  time  series  methods  and 
causal  models.  Time  series  techniques  are  beneficial  when  several  years  of  data  are  available 
and  helps  identify  and  explain  seasonality,  cyclical  patterns,  trends,  or  growth  rates  of  the  trends 
within  the  data.  Casual  modeling  is  the  most  mathematically  complex  fonn  of  forecasting  and 
analysis.  Mathematics  expresses  relationships  within  the  data  or  between  different  variables  and 
can  account  for  cause  and  effect  relationships  from  events  as  well.  (Chamber,  Mullick,  &  Smith, 
1971)  For  the  issue  at  hand,  complexity  is  necessary  in  order  to  increase  the  accuracy  of  the 
forecast  and  to  ensure  that  the  historical  data  could  produce  beneficial  results. 

Although  the  current  problem  is  directly  related  to  airlifting  cargo,  there  are  many  other 
examples  of  forecasted  demand  using  the  same  techniques.  Studies  have  been  conducted  using 
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demand  forecasting  for  semiconductor  companies  to  detennine  investment  and  manufacturing 
strategies  for  capacity  planning.  Semiconductors  have  no  relation  to  cargo,  but  as  shown,  time 
series  methods  have  been  applied  in  many  companies  that  are  increasingly  dependent  on  future 
demands.  The  semiconductor  research  proposed  using  a  multi-generation  diffusion  model  that 
involving  multiple  input  factors  that  changed  across  technological  generations  (Chien,  Chen,  & 
Peng,  2008).  The  complexity  and  broadness  of  this  type  of  model  is  not  necessary  for  this 
problem,  but  presents  a  very  interesting  approach  that  provides  good  fitting  models  and  precise 
forecasts. 

Forecasting  cargo  demand  is  quite  common.  Companies  such  as  Federal  Express  use 
forecasting  models  to  forecast  their  cargo  growth  into  the  future  to  plan  for  acquisitions  and  site 
expansions  (Datamonitor,  2011).  Many  states  provide  basic  forecasts  of  cargo  shipments  into 
their  state  as  a  measure  of  potential  profitability.  California  and  Florida  specifically  have  done 
extensive  studies  about  the  future  of  air  cargo  tonnage  in  their  states  and  the  impact  it  presents  on 
their  current  airport  capabilities  (TranSystems,  2010;  Air  Cargo  Tonnage  Forecast  and  Capacity 
Analysis,  2005).  However,  these  analyses  only  look  at  past  trends  and  uses  naive  forecasts  based 
on  numbers  from  the  shipping  companies. 

More  advanced  air-cargo  demand  forecasting  has  been  approached  in  recent  years  as 
well,  as  it  is  one  of  the  key  issues  of  airline  revenue  management.  A  new  technique  called 
support  vector  machine  (SVM)  model  was  applied  to  cargo-volume  data  from  Beijing  and  then 
compared  Brown’s  Cubic  exponential  smoothing  model  (Heng,  Zheng,  &  Li,  2009).  The 
accuracy  of  the  SVM  forecasting  model  is  more  accurate  than  Brown’s  model;  however,  more 
research  needs  to  be  conducted  in  the  application  of  structure  based  learning  models  to  detennine 
if  SVM  is  generally  better  at  time  series  prediction.  Although  this  technique  may  be  useful  in  the 
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future,  a  black  or  grey  box  approach  is  not  as  valuable  to  the  user  at  this  time  due  to  a  more 
difficult  implementation. 

More  traditional  time  series  forecasting  approaches  as  mentioned  in  the  previous  section 
are  easily  implemented  and  automated.  As  we  show  in  the  next  section,  these  types  of  models 
are  also  quite  precise  as  well.  The  research  did  not  provide  a  direct  application  of  exponential 
smoothing  methods  or  ARIMA  methods  used  on  airlift  or  cargo  demand,  but  still  produced  other 
forecasting  demand  applications  that  are  just  as  beneficial.  Time-series  forecasting  using 
ARIMA  and  SARIMA  methods  to  estimate  future  primary  energy  demand  of  Turkey  from  2005 
to  2020  have  been  shown  to  be  reliable  methods  (Ediger  &  Akar,  2007).  Energy  forecasting 
dates  back  to  the  1960s  and  recent  studies  have  applied  regression,  auto  regression,  genetic 
algorithms,  ARIMA,  SARIMA,  and  neural  network  methods  to  historical  data.  Research  has 
shown  that  the  parameters  used  in  these  various  methodologies  usually  deviate  from  the  actual 
values,  making  the  forecasts  unreliable.  By  using  ARIMA  or  SARIMA,  the  additional 
parameters  can  be  eliminated  and  the  energy  demand  can  be  estimated  by  its  own  time  series. 
The  final  results  showed  that  the  ARIMA  and  the  SARIMA  models  were  both  efficient  and 
reliable  for  forecasting  of  energy  demand  based  on  the  type  of  energy. 

Further  research  provides  an  excellent  comparison  of  SARIMA  and  artificial  neural 
networks  (ANN)  to  forecast  cement  demand  (Liu,  Chen,  Yang,  Hung,  &  Tsai,  2008).  The  study 
compares  the  forecast  accuracy  of  the  two  methods.  ANN  is  a  complex  modeling  approach  that 
most  mathematical  or  statistical  packages  now  implement.  The  black  box  work  of  an  ANN 
involves  continuously  adjusting  weighted  values  applied  to  the  inputs  until  the  network  output 
reaches  a  target  value.  The  conclusion  was  that  the  ANN  can  provide  a  more  accurate  demand 
forecast  without  the  addition  of  functions  to  account  for  outside  influences.  The  SARIMA 
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performed  well,  but  needs  possible  modification  to  an  intervention  model  to  account  for 
holidays,  weather,  and  economic  cycles. 

Summary 

There  is  no  single  best  forecasting  method.  The  selection  of  the  appropriate  model  has 
many  determining  criteria  and  is  subjective  to  the  output,  the  available  information,  and  the  user. 
The  literature  has  expressed  that  forecasting  is  both  an  art  and  a  science  and  the  development  of 
the  model  depends  on  both  the  analytical  steps  developed  and  what  constitutes  a  pattern  or  a 
significant  feature.  This  study  compares  forecasting  techniques  through  application  and  provides 
analysis  as  to  which  method  is  preferred  for  the  problem  of  predicting  cargo  demand. 
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III.  Methodology 

This  section  discusses  the  development  of  the  forecasting  models  and  the  cargo 
scheduling  heuristic  constructed  to  simulate  Dover  AFB  aerial  port  airlift  to  Afghanistan. 

Scope  and  Data  Description 

The  TACC  Strategic  Airlift  Simulation  (TACCSAS)  is  constructed  as  a  tool  to  test  the 
usefulness  of  previously  constructed  airlift  schedules  and  to  forecast  future  demand  and  evaluate 
potential  airlift  schedules.  TACCSAS  allows  the  user  to  simulate  scheduling  cargo  pallets  and 
airlift  for  a  specific  month  based  on  previously  constructed  schedules,  or  to  forecast  a  full  year, 
one  quarter,  or  a  single  month  ahead.  All  pallets  travel  from  Dover  AFB  using  CRAF  aircraft 
and  Organic  military  aircraft  previously  purchased  or  scheduled  for  that  month.  The  user  selects 
the  period  that  they  would  like  to  simulate  and  then  selects  the  simulation  parameters  to  include 
number  of  runs  and  cargo  pallet  weight  distribution  parameters.  The  user  determines  available 
numbers  of  aircraft  types,  can  add  and  delete  aircraft  available  for  transport,  change  aircraft 
capability  parameters,  and  add  or  delete  scheduled  aircraft  departures.  Once  the  simulation  has 
run,  the  user  can  view  and  compare  the  collected  statistics.  TACCSAS  was  constructed  entirely 
using  Microsoft  Excel  Visual  Basic  for  Applications  (VBA)  (Microsoft,  2007).  Examples  of 
TACCSAS  operations  and  further  description  are  provided  in  Appendix  E. 

The  data  used  for  forecasting  is  provided  by  GATES.  GATES  is  an  automated  aerial  port 

processing  system  that  aids  in  scheduling  of  unit  and  cargo  movement  and  shipment  forecasting. 

(Westcott,  2006)  The  GATES  data  sets  are  generated  by  fiscal  year  (FY)  and  include  31 

categories  by  which  to  sort  the  data.  These  categories  include  pallet  ID  number,  date  of  pallet 

arrival,  gross  pallet  weight,  mission  ID  numbers,  aerial  port  of  departure  (APOD),  aerial  port  of 

embarkation  (APOE),  and  departure  date  among  many  others.  To  ensure  adequate  models  could 
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be  developed,  GATES  reports  for  FY2005  through  FY2010  are  split  into  monthly  cargo  totals 
shipped  into  Afghanistan.  Pallet  data  from  FY201 1  is  separated  and  used  validation.  The  issue 
with  the  GATES  data  outputs  is  that  the  cargo  report  generated  includes  all  cargo  entries  from  all 
APOEs  to  all  APODs  for  every  mission  type.  This  allows  for  FY  report  to  include  anywhere 
from  275,000  to  325,000  individual  entries.  Figure  1 1  shows  the  mission  character  codes 
observed  and  filtered  to  simplify  the  data  set.  Under  Second  Character,  only  channel  cargo  is 
observed,  so  all  mission  IDs  including  the  character  “B”  are  retained.  Under  Third  Character,  all 
missions  are  retained  except  those  IDs  that  include  “C”  or  “P.” 


Second  Character 

Third  Character 

B 

Channel  Cargo 

A 

Not  Assigned* 

J 

Positioning  to  first  onload 

B 

Distribution  Channel,  Atlantic  Region* 

K 

Channel  PAX 

C 

Distribution  Channel,  Pacific  Region* 

L 

Aeromedical  Evacuation  (AE) 

P 

Not  Assigned* 

Q 

Channel  Mixed  (PAX  and  cargo) 

E/Q 

CPX  Channel  missions  as  assigned  by 

18  AF/A3Y 

V 

Depositiouing  from  offload  to  new  mission  or 
home  station 

J/R/U 

W/Y/Z 

Channel  missions  supporting  Contingency 
Operations.  Coordinate  ID  with  618 

AOC/XOP 

Figure  11  AMC  Mission  Character  Codes  (AMC/A3CF,  2012) 

The  research  focuses  on  cargo  channel  missions  directed  to  Afghanistan.  To  reduce  the 
data  set  even  further,  only  entries  that  departed  from  Dover  AFB  and  arrived  in  Afghanistan  are 
considered.  The  data  set  includes  repeat  pallet  entries,  which  occurred  whenever  an  individual 
pallet  was  transferred  to  a  different  aircraft.  Only  considering  the  pallets  that  entered  into 
Afghanistan  filtered  out  the  repeated  pallet  entries.  The  data  were  then  sorted  by  month  and 
summed  into  a  single  tonnage  scheduled  for  that  month.  The  monthly  totals  are  used  to  generate 
the  forecasting  models.  Monthly  totals  are  used  to  accommodate  the  monthly  BOG  input. 
Weekly  BOG  values  were  unattainable. 
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Model  Selection 


All  of  the  approaches  described  in  Chapter  2  are  applied  to  the  problem.  The  result  is  a 
detailed  comparison  of  1 1  different  models. 

Chapter  2  presented  a  number  of  different  criteria  to  evaluate  the  goodness  of  fit  of  the 
models  that  are  developed.  Below  is  a  detailed  chart  of  9  of  the  1 1  models  that  are  developed  as 
are  summarized  by  JMP.  The  two  regression  models  are  not  included  in  the  chart  due  to  the 
limitations  of  the  JMP  program.  All  of  the  models  are  good  models  according  to  the  criteria 
presented.  The  smoothing  methods  provided  the  best  results  and  the  next  best  are  the  transfer 
functions.  The  standard  ARIMA  models  had  too  much  variance  between  the  predicted  and  the 
actual  values. 


If  it  were  only  a  matter  of  choosing  the  best,  one  would  easily  choose  Transfer  Function 
Model  (1)  as  its  criteria  are  better  in  most  aspects.  However,  the  best  way  to  evaluate  a  model  is 
validation  via  data  splitting.  Data  splitting  divides  the  data  into  two  segments,  one  to  fit  the 
model  and  one  to  evaluate  the  model  (Montgomery,  Jennings,  &  Kulahci,  2008).  The  benefit  of 
this  technique  is  the  knowledge  of  how  the  model  will  perform  on  new  data  and  to  compare 


competing  models.  Data  splitting  is  perfonned  on  the  12  cargo  values  for  FY1 1.  All  1 1  models 


provided  12  month  forecasts. 


Model 

DF 

Variance 

AIC 

SBC 

RSquare 

-2LogLH 

MAPE 

MAE 

—  Seasonal  Exponential  Smoothing(  12,  Zero  to  One ) 

54 

285043.11 

885.17672 

889.22743 

0.800 

881.17672 

25.763687 

460.09723 

—  Winters  Method  (Additive) 

53 

290421.28 

887.17672 

893.25278 

0.800 

881.17672 

25.763687 

460.09723 

—  Transfer  Function  Model  (1) 

63 

271794.88 

1032.7003 

1041.5191 

0.851 

1024.7003 

21.399069 

344.39029 

—  Transfer  Function  Model  (2) 

63 

278510.57 

1034.3099 

1043.1287 

0847 

1026.3099 

22.926044 

363.81241 

—  Double  (Brown)  Exponential  Smoothing 

66 

363739.08 

1051.0052 

1053.2099 

0.780 

1049.0052 

23.919009 

428.18849 

—  Simple  Exponential  Smoothing(Zero  to  One ) 

67 

320377.63 

1056.1085 

1058.3280 

0.813 

1054.1085 

22.196737 

392.09033 

—  IMA(1, 1) 

66 

321788.61 

1057.4012 

1061.8402 

0.815 

1053.4012 

23.179941 

394.41567 

—  ARIMA(1, 1,1) 

65 

319613.19 

1058.0312 

1064.6897 

0.819 

1052.0312 

23.817296 

391.03950 

—  ARI(1, 1) 

66 

327632.66 

1058.5760 

1063.0150 

0.812 

1054.576 

23.031031 

397.05670 

—  Seasonal  ARIMA(0, 1, 1X1, 0, 1)12 

64 

290546.15 

1059.3699 

1068.2479 

0.822 

1051.3699 

23.096380 

389.21300 

Figure  12  JMP  Output  -  Model  Selection  Criteria 
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Data  splitting  may  not  always  be  an  option.  The  purpose  of  forecasting  is  to  provide  a 
prediction  for  the  future,  often  times  eliminating  the  ability  to  validate  results.  For  this  reason, 
goodness-of-fit  criteria  provide  standardized  evaluation  methods  to  determine  which  model  best 
fits  the  data.  In  addition  to  the  criteria,  one  must  apply  common  sense  to  the  evaluation.  If  the 
model  maps  the  data  set  near  perfectly,  but  provides  extreme  forecasts,  then  the  criteria  are  not 
useful  and  neither  is  the  model. 

Table  2  indexes  each  model  developed.  Throughout  the  model  selection  process  each 
model  is  referenced  by  its  specific  number. 


Model  1 

Linear  Regression 

Model  2 

Intervention 

Model  3 

Simple  Exponential  Smoothing 

Model  4 

Double  Exponential  Smoothing 

Model  5 

Seasonal  Exponential  Smoothing 

Model  6 

ARIMA(  1,1,1) 

Model  7 

ARI  ( 1, 1) 

Model  8 

1  MA(  1,1) 

Model  9 

Seasonal  ARIMA  (0, 1, 1) ( 1,0, 1)  12 

Model  10 

ARIMA(1,0,0)  /  ARIMA(0,1,1) 

Model  11 

ARIMA(0,0,1)  /  ARIMA(0,1,1) 

Table  2  Model  Number  Designators 


To  select  the  best  fitting  model  among  such  similarly  valued  selection  criteria,  data 
splitting  is  employed  by  holding  out  FY201 1  cargo  demand  data  from  the  forecast.  A  12-month 
forecast  is  conducted  for  each  model  and  then  compared  with  the  actual  FY201 1  results  shown  in 
Table  2.  The  residuals  are  then  used  to  compute  the  percentage  deviation  from  the  actual  results 
shown  in  Table  3.  The  research  shows  that  some  of  the  models  only  predict  the  first  month  and 
use  that  value  for  all  of  the  following  forecasts.  Other  models  have  adjusted  values  for  each 
month.  Some  of  these  models  incorporate  some  degree  of  seasonality  in  the  data,  whether  from 
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the  Tons  of  Cargo  or  from  the  BOG  input.  There  is  no  expectation  that  the  model  can  accurately 
predict  a  full  years  worth  of  cargo  demand,  but  a  fair  estimate  is  necessary.  The  critical  forecasts 
are  the  ability  to  predict  a  quarterly  buy  (3  month  forecast)  or  a  monthly  buy  (1  month  forecast). 
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Date 

Tons  of  Cargo 

Model  1 

Model  2 

Model  3 

Model  4 

Model  5 

Model  6 

Model  7 

Model  8 

Model  9 

Model  10 

Model  11 

Oct- 10 

3,300.60 

4,922.26 

4,074.61 

3,306.13 

3,187.91 

3,635.69 

3,512.38 

3,235.07 

3,399.52 

3,503.98 

3,510.47 

3,635.99 

Nov- 10 

4,149.92 

4,922.26 

4,540.01 

3,306.13 

2,924.94 

3,520.29 

3,779.81 

3,227.67 

3,439.99 

3,481.50 

3,725.43 

4,090.68 

Dec- 10 

4,536.60 

4,922.26 

4,862.79 

3,306.13 

2,661.98 

3,619.96 

3,930.55 

3,276.24 

3,480.45 

3,547.50 

3,785.59 

3,940.11 

Jan- 11 

5,069.22 

4,922.26 

5,082.43 

3,306.13 

2,399.01 

3,752.85 

4,025.93 

3,311.95 

3,520.92 

3,624.14 

3,765.92 

3,797.04 

Feb-11 

4,852.68 

4,922.26 

4,929.59 

3,306.13 

2,136.05 

3,783.84 

4,095.04 

3,350.61 

3,561.38 

3,661.37 

3,706.01 

3,661.09 

Mar-11 

4,529.13 

4,922.26 

5,040.70 

3,306.13 

1,873.08 

4,480.50 

4,151.70 

3,388.59 

3,601.85 

4,002.25 

3,626.78 

3,531.91 

Apr-11 

4,037.11 

4,922.26 

5,232.21 

3,306.13 

1,610.12 

4,461.58 

4,202.45 

3,426.73 

3,642.31 

4,016.71 

3,539.20 

3,409.17 

May-11 

4,370.84 

4,922.26 

5,042.36 

3,306.13 

1,347.15 

4,284.65 

4,250.39 

3,464.84 

3,682.78 

3,959.10 

3,449.01 

3,292.55 

Jun-11 

5,318.32 

4,922.26 

4,982.95 

3,306.13 

1,084.19 

4,401.08 

4,297.01 

3,502.95 

3,723.24 

4,035.31 

3,359.19 

3,181.73 

Jul-11 

3,691.94 

4,922.26 

4,863.00 

3,306.13 

821.22 

4,349.26 

4,342.99 

3,541.06 

3,763.71 

4,034.77 

3,271.24 

3,076.43 

Aug-11 

4,014.77 

4,922.26 

4,734.02 

3,306.13 

558.26 

4,295.82 

4,388.68 

3,579.17 

3,804.17 

4,033.49 

3,185.88 

2,976.38 

Sep- 11 

4,941.08 

4,922.26 

4,423.57 

3,306.13 

295.29 

3,732.69 

4,434.22 

3,617.28 

3,844.64 

3,799.72 

3,103.44 

2,881.31 

Table  3  Actual  Tons  of  Cargo  vs.  Predicted  Tons  of  Cargo 


Date 

Tons  of  Cargo 

Model  1 

Model  2 

Model  3 

Model  4 

Model  5 

Model  6 

Model  7 

Model  8 

Model  9 

Model  10 

Model  11 

Oct- 10 

3,300.60 

49.13% 

23.45% 

0.17% 

3.41% 

10.15% 

6.42% 

1.99% 

3.00% 

6.16% 

6.36% 

10.16% 

Nov-10 

4,149.92 

18.61% 

9.40% 

20.33% 

29.52% 

15.17% 

8.92% 

22.22% 

17.11% 

16.11% 

10.23% 

1.43% 

Dec-10 

4,536.60 

8.50% 

7.19% 

27.12% 

41.32% 

20.21% 

13.36% 

27.78% 

23.28% 

21.80% 

16.55% 

13.15% 

Jan- 11 

5,069.22 

2.90% 

0.26% 

34.78% 

52.67% 

25.97% 

20.58% 

34.67% 

30.54% 

28.51% 

25.71% 

25.10% 

Feb-11 

4,852.68 

1.43% 

1.58% 

31.87% 

55.98% 

22.03% 

15.61% 

30.95% 

26.61% 

24.55% 

23.63% 

24.56% 

Mar-11 

4,529.13 

8.68% 

11.30% 

27.00% 

58.64% 

1.07% 

8.33% 

25.18% 

20.47% 

11.63% 

19.92% 

22.02% 

Apr- 11 

4,037.11 

21.93% 

29.60% 

18.11% 

60.12% 

10.51% 

4.10% 

15.12% 

9.78% 

0.51% 

12.33% 

15.55% 

May- 11 

4,370.84 

12.62% 

15.36% 

24.36% 

69.18% 

1.97% 

2.76% 

20.73% 

15.74% 

9.42% 

21.09% 

24.67% 

Jun-11 

5,318.32 

7.45% 

6.31% 

37.84% 

79.61% 

17.25% 

19.20% 

34.13% 

29.99% 

24.12% 

36.84% 

40.17% 

Jul-11 

3,691.94 

33.32% 

31.72% 

10.45% 

77.76% 

17.80% 

17.63% 

4.09% 

1.94% 

9.29% 

11.40% 

16.67% 

Aug- 11 

4,014.77 

22.60% 

17.91% 

17.65% 

86.09% 

7.00% 

9.31% 

10.85% 

5.25% 

0.47% 

20.65% 

25.86% 

Sep-11 

4,941.08 

0.38% 

10.47% 

33.09% 

94.02% 

24.46% 

10.26% 

26.79% 

22.19% 

23.10% 

37.19% 

41.69% 

Table  4  Percentage  Deviation  from  Actual  Tons  of  Cargo 


It  can  be  difficult  to  visualize  the  differences  between  the  models  when  looking  at  a  chart 
of  numerical  values  or  percentages.  The  figures  below  provide  a  visual  representation  of  all  of 
the  models  predicted  values  compared  to  the  actual  tons  of  cargo.  The  benefit  of  the  charts  is 
that  one  can  see  how  closely  the  models  follow  the  entire  five  year  sequence  of  actual  values  as 
well  as  the  disadvantages  of  the  forecasted  values.  It  is  quite  apparent  which  models  attempt  to 
forecast  with  any  degree  of  accuracy  and  which  models  are  only  capable  of  providing  a  one-step 
ahead  forecast.  Seasonality  can  be  important  in  forecasting  as  it  allows  for  increases  and 
decreases  correlated  with  certain  seasonal  trends.  Unfortunately,  in  some  models  the  seasonality 
can  sway  the  predictions  in  the  wrong  direction  based  on  prior  trends.  Judging  by  appearance, 
there  appears  to  be  one  stronger  model  or  better  estimating  model  that  stands  out  in  each  chart  in 
Figure  13,  Figure  14,  and  Figure  15.  The  transfer  function  models  have  the  worst  predictive 
capability  in  this  case  (Figure  16). 


-•-Linear  Regression 

-■-Intervention  \  _  * 

Ions ot Largo  1  ]r€***/T\*i  ^ 

oe°'° 

A,cfr  KJ$  <JGfl  <J0l&  AjOi&  v,0%  y.,09  dJDp 

q9  ,  \0  \0  \0  VV  VV  .  VV  VV 

oec'  c,eV'v  y^'v  ^o'4' 

Figure  13  Tons  of  Cargo  vs.  Regression  Models 
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-*-Simple  Exponential  Smoothing 


Figure  14  Tons  of  Cargo  vs.  Exponential  Smoothing  Models 


Figure  15  Tons  of  Cargo  vs.  ARIMA  Models 
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The  three  month  forecast  is  represented  below  in  Table  5.  The  field  of  models  is 
narrowed  down  by  observing  the  percentage  deviation  on  a  three  month  forecast.  The  absolute 
deviation  measures  the  accuracy  of  the  prediction.  Percent  deviation  is  equal  to  |Actual- 
Forecast|/Actual.  For  this  forecasting  application,  it  is  not  as  important  to  measure  the  increase 
or  decrease  of  the  forecast  from  the  actual  value.  If  the  forecast  is  too  high,  too  many  aircraft  are 
reserved  and  cancellations  will  occur,  and  the  cost  to  schedule  will  be  incurred.  If  the  forecast  is 
too  low,  then  “expansion  buys”  will  have  to  be  completed  and  additional  money  will  be  spent. 
The  deviations  in  the  forecasts  in  Table  5  appear  to  steadily  increase  due  to  larger  increases  from 
October  2010  to  December  2010  in  the  data  set.  In  this  situation,  planning  would  remain  slightly 
ad  hoc,  but  a  foundation  of  scheduled  aircraft  can  still  be  established  from  the  forecast.  Once  the 
next  data  point  is  included  into  the  data  set,  all  models  need  to  be  rerun  and  a  new  best  fitting 
model  selected.  Rerunning  the  models  with  all  available  data  includes  the  most  recent  changes 
and  the  better  models  will  begin  to  react  appropriately.  The  downfall  to  the  simulation  is  if  the 
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forecasts  happen  to  be  low  for  the  upcoming  months,  then  the  planners  make  more  purchases  for 
the  monthly  buy. 


Actual  vs  Predicted 


Date 

Tons  of  Cargo 

Model  1 

Model  2 

Model  5 

Model  6 

Model  8 

Model  9 

Oct- 10 

3,300.60 

4,922.26 

4,074.61 

3,635.69 

3,512.38 

3,399.52 

3,503.98 

Nov- 10 

4,149.92 

4,922.26 

4,540.01 

3,520.29 

3,779.81 

3,439.99 

3,481.50 

Dec- 10 

4,536.60 

4,922.26 

4,862.79 

3,619.96 

3,930.55 

3,480.45 

3,547.50 

Resduais 


Date 

Tons  of  Cargo 

Model  1 

Model  2 

Model  5 

Model  6 

Model  8 

Model  9 

Oct- 10 

3,300.60 

(1,621.66) 

(774.01) 

(335.09) 

(211.79) 

(98.92) 

(203.38) 

Nov- 10 

4,149.92 

(772.34) 

(390.09) 

629.62 

370.11 

709.93 

668.42 

Dec- 10 

4,536.60 

(385.65) 

(326.19) 

916.64 

606.06 

1,056.15 

989.10 

Percentage  Deviation 


Date 

Tons  of  Cargo 

Model  1 

Model  2 

Model  5 

Model  6 

Model  8 

Model  9 

Oct- 10 

3,300.60 

49.13% 

23.45% 

10.15% 

6.42% 

3.00% 

6.16% 

Nov- 10 

4,149.92 

18.61% 

9.40% 

15.17% 

8.92% 

17.11% 

16.11% 

Dec- 10 

4,536.60 

8.50% 

7.19% 

20.21% 

13.36% 

23.28% 

21.80% 

Average  %  Deviation  25.41%  13.35%  15.18%  9.56%  14.46%  14.69% 


Table  5  Quarterly  Buy  Forecast  Comparison 


The  one  month  forecast  is  represented  in  Table  6.  Since  all  of  the  models  are  similar  in 
their  predictive  ability,  with  the  exception  of  Model  1 ,  the  same  set  of  five  models  are  compared 
for  the  one  month  forecast.  All  of  the  models  are  very  close  when  it  comes  to  one  month 
forecasting,  however,  the  Model  8  stands  out  above  the  rest.  Further  data  splitting  analysis  is 
conducted  on  the  five  remaining  models  in  Chapter  4. 
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Actual  vs  Predicted 


Date 

Tons  of  Cargo 

Model  2 

Model  5 

Model  6 

Model  8 

Model  9 

Oct- 10 

3,300.60 

4,922.26 

3,635.69 

3,512.38 

3,399.52 

3,635.99 

Resduals 


Date 

Tons  of  Cargo 

Model  2 

Model  5 

Model  6 

Model  8 

Model  9 

Oct- 10 

3,300.60 

(774.01) 

(335.09) 

(211.79) 

(98.92) 

(203.38) 

Percentage  Deviation 


Date 

Tons  of  Cargo 

Model  2 

Model  5 

Model  6 

Model  8 

Model  9 

Oct- 10 

3,300.60 

23.45% 

10.15% 

6.42% 

3.00% 

6.16% 

Table  6  Monthly  Buy  Forecast  Comparison 


Modeling  Approach 

The  modeling  approach  was  initially  a  reverse  concept  to  determine  whether  or  not 
optimization  of  CRAF  on  a  monthly  basis  was  necessary  and  to  determine  what  the  cost  savings 
are.  In  this  case,  cost  savings  are  determined  by  the  number  of  cancellations  generated  on  a 
monthly  basis.  As  described  in  Chapter  1,  it  may  seem  counterintuitive  that  savings  involve 
paying  to  cancel  a  plane;  however,  the  cancellation  cost  is  only  ~20%  of  the  flight  cost.  If  cargo 
loading  is  optimized  on  specific  types  of  CRAF  airframes,  then  it  is  easy  to  calculate  the 
approximate  number  of  aircraft  necessary  to  move  a  certain  number  of  cargo  pallets  or  cargo 
tons.  The  simulated  number  of  aircraft  used  to  airlift  the  cargo  pallets  is  then  compared  to  the 
actual  number  of  aircraft  used,  with  the  difference  in  numbers  turning  into  immediate  savings. 

This  report  previously  mentions  the  use  of  simulation  for  operational  planning, 
specifically  Monte  Carlo  simulation.  Monte  Carlo  simulation  is  a  method  of  analysis  (usually 
using  a  computer)  to  recreate  a  random  process  many  times  in  a  model  of  a  real  system  to  study 
and  understand  the  system  (Barreto  &  Howland,  2006). 

Fortunately,  an  initial  framework  of  the  Monte  Carlo  simulation  was  constructed  by 
Lindstrom  in  his  Graduate  Research  Project  (GRP)  (Lindstrom,  2012).  The  previous  solution 
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was  reworked  into  TACCSAS  to  provide  quick  answers  by  selecting  a  few  options.  By 
reworking  the  problem  from  the  beginning,  the  new  system  is  now  more  robust  and  flexible  to 
meet  the  needs  of  TACC. 

To  simulate  the  actual  process,  Monte  Carlo  simulations  of  a  selected  month  are  run 
using  randomized  parameters  calculated  based  on  distributions  modeling  the  actual  pallet  data. 
The  real  time  data  required  to  perfonn  the  operational  planning  aspect  is  the  pallet  data.  The 
weight  of  the  pallet,  the  destination  of  the  pallet  and  the  arrival  date  of  the  pallet  to  the  base  are 
the  three  important  components.  These  three  parameters  are  randomly  produced  with  each  run  to 
fill  a  realistic  schedule.  The  randomized  weights  of  the  pallets  are  provided  by  selecting  one  of 
the  following  distributions:  Exponential,  Lognonnal,  Normal,  Weibull,  or  Triangular.  Once  a 
distribution  is  selected,  the  distribution  parameters  from  the  past  pallet  data  from  the  selected 
month  are  calculated  and  used  as  the  basis  for  the  random  weight  distribution.  For  example,  if 
the  nonnal  distribution  is  selected  to  simulate  the  pallet  weight  from  October,  then  the  mean  and 
the  standard  deviation  from  the  past  October’s  pallet  data  is  calculated  and  used  to  produce  a 
predetermined  number  of  random  pallet  weights.  A  random  destination  location  is  then  assigned 
to  each  individual  pallet  based  off  of  the  distribution  of  the  actual  destination  locations  from  the 
selected  month.  For  example,  the  month  of  October  may  show  that  40%  of  the  actual  scheduled 
pallets  are  destined  for  Kandahar,  30%  to  Bagram,  and  30%  to  Bastion;  the  pallet  locations  are 
then  randomly  assigned  to  follow  this  discrete  distribution.  A  random  date  is  also  assigned  to 
each  pallet  simulated.  The  dates  are  drawn  from  a  discrete  distribution  derived  from  the  actual 
past  month’s  pallet  data.  A  sample  of  the  actual  aircraft  schedule,  actual  pallet  schedule,  discrete 
distribution,  and  Monte  Carlo  Simulation  is  viewed  in  Appendix  D.  These  three  components 
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then  make  up  a  simulated  full  month  of  pallets  that  are  sorted  by  date,  then  location,  and  are 
assigned  to  individual  aircraft. 

Sometimes  in  Operations  Research,  there  are  problems  that  require  too  much 
computational  time  to  solve  exactly  or  may  require  an  approximate  solution  method  due  to  the 
nature  of  the  problem.  In  these  cases,  constructive  methods  known  as  heuristics  are  useful  for 
finding  good,  yet  not  exact,  solutions.  Given  more  time  and  more  fidelity  in  the  data  available, 
an  optimal  solution  may  have  been  possible  for  the  problem,  however,  heuristics  methods  have 
allowed  for  a  good  feasible  solution  with  room  for  sensitivity  analysis.  There  are  many  benefits 
to  heuristic  methods;  the  biggest  may  be  that  they  tend  to  be  common-sense  approaches  that  are 
tailored  to  the  specific  problem  (Metaheuristics,  2005).  As  seen  in  the  section  Cargo  Loading 
Algorithm,  heuristic  methods  are  often  iterative  algorithms  where  each  iteration  conducts  a  new 
search  for  a  better  solution. 

After  the  heuristic,  known  as  the  cargo  loading  algorithm,  was  developed  for  the  pallet 
assignment,  the  forecasting  portion  was  incorporated  to  allow  for  the  operational  planning  aspect 
of  the  problem  in  addition  to  the  sensitivity  analysis  portion. 

Model  Inputs 

The  first  input  required  is  the  cargo  tonnage  forecast  generated  from  the  cargo  forecast 
models.  The  model  requires  an  estimated  number  of  pallets  for  a  single  month  period.  The 
number  of  pallets  is  derived  from  the  weight  distribution  assigned  to  the  cargo  pallets.  For 
example,  if  TACC  will  airlift  3,000  tons  of  cargo  in  January  to  Afghanistan,  and  the  average 
pallet  weighs  2  tons,  then  the  simulation  input  is  1,500  pallets.  The  pallet  weight  is  then 
normally  distributed  with  the  mean  and  the  variance  generated  by  the  GATES  data  and  using  the 
minimum  pallet  weight  as  a  floor  for  the  distribution  to  prevent  negative  pallet  weights.  The 
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model  then  assigns  the  pallets  to  the  existing  CRAF  airframes  in  the  current  inventory  based  on 
the  route  that  the  airframes  are  assigned.  Once  all  the  pallets  are  assigned  or  once  all  of  the 
available  aircraft  are  filled  either  with  minimum  number  of  pallets  or  by  the  minimum  number  of 
tons,  the  problem  completes  and  the  statistics  (averaged  by  number  of  runs)  are  displayed. 

Cargo  Loading  Algorithm 

The  main  objective  of  the  problem  is  to  move  as  much  cargo  as  possible  as  efficiently  as 
possible.  The  ACL  constraints  allow  for  the  additional  objectives  of  minimizing  the  number  of 
aircraft  used  and  minimizing  the  cost  of  the  airlift  schedule  achieved  in  the  process.  Due  to  the 
current  operations  of  USTRANSCOM,  the  algorithm  fills  CRAF  aircraft  flying  directly  to  an 
AOR  location  first  before  filling  Organic  aircraft  going  to  the  same  location  on  the  same  day. 

For  this  research,  cargo  with  an  assigned  destination  is  only  loaded  onto  an  aircraft  with  the  same 
destination. 

The  algorithm  is  initialized  after  receiving  three  initial  inputs:  the  weight  of  the 
individual  pallets,  the  APOD  of  the  pallet,  and  the  date  of  departure  from  DOVER  AFB.  Once 
received,  either  through  simulated  data  or  from  the  actual  cargo  data,  the  first  phase  of  the 
algorithm  assigns  pallets  to  available  CRAF.  The  algorithm  examines  pallets  in  the  sorted  order 
of  date  available,  then  APOD.  It  then  detennines  if  any  CRAF  aircraft  are  flying  on  that  date 
and  going  to  that  APOD.  The  pallets  available  for  shipping  on  that  day  or  that  had  not  yet  been 
shipped  the  previous  days  to  the  same  location  are  assigned  to  the  available  aircraft.  Loading 
pallets  continues  until  the  weight  limit  is  met  or  until  all  of  the  pallets  slots  are  filled.  This 
process  is  repeated  until  all  aircraft  available  for  flying  on  that  date  are  filled  to  capacity  or  until 
all  of  the  pallets  have  been  loaded,  whichever  occurs  first. 
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If  there  are  no  CRAF  aircraft  available  for  airlift  on  that  specific  day  and  going  to  that 
same  location,  the  algorithm  then  moves  to  the  second  phase  to  determine  if  there  are  any  CRAF 
available  flying  to  enroute  locations.  If  there  are  no  enroute  CRAF  flights  available  on  that  day, 
the  algorithm  moves  to  the  third  and  final  phase  to  detennine  if  any  organic  military  aircraft  are 
available  on  that  date  and  going  to  that  APOD.  The  organic  military  aircraft  are  the  final  attempt 
to  transport  skipped  pallets.  If  there  are  no  more  aircraft  to  accommodate  the  available  pallets, 
then  the  pallets  remain  on  site  and  are  moved  later. 

The  general  outline  of  the  objectives  and  the  constraints  for  the  process  are  shown  below. 
The  objectives  apply  to  the  problem  as  a  whole,  while  the  constraints  are  applied  in  an  iterative 
process  checks  each  of  these  constraints  in  the  order  of  the  subscripts. 


For  each  i  =  1  to  Days  in  Month  (ex.  Jan  =  31  days) 

For  each  j  =  1  to  Location  Number  (ex.  1  =  OAKN) 

For  each  k  =  1  to  Number  of  Aircraft  Types  (ex.  B747-200  is  one  type) 

If  current  aircraft  =  "CRAF"  and  current  aircraft  flies  to  j  and  is  available  on  i  Then 

While  Total  tons  on  aircraft  <  MaxTons  And  Total#  Pallets  on  Aircraft  <  MaxPallets 
For  each  pallet  not  yet  shipped  by  date  i  going  to  location  j 

Increment  Total  Tons  and  Total  #  Pallet 

If  Total  tons  on  aircraft  >  MinTons  Or  Total  #  Pallets  on  Aircraft  >  MinPallets  Then 
Then  schedule  loaded  aircraft 
If  aircraft  is  not  loaded,  then  cancel 

Figure  17  Cargo  Loading  Algorithm 
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System  Output 

TACCSAS  provides  individual  run  outputs  and  outputs  averaged  across  all  runs  of  the 
simulation.  The  averaged  statistics  shown  in  Table  7  are  classified  by  airframe  and  show 
average  allocation  of  pallets  and  utilization  of  the  airframe. 


Simulated  Pallet  Statistics 

#  missions 

Tonnage 

Tons/msn 

pit  offer 

pallets  used 

%  pallets  utilized 

Avg  pallet  wt 

B74710 

19 

1248.21 

65.70 

798.00 

192.20 

24.1% 

6.51 

B74720 

45.8 

4945.86 

107.99 

3068.60 

765.00 

24.9% 

6.47 

B74740 

7.30 

884.68 

121.18 

540.20 

130.60 

24.2% 

6.78 

C005A 

6.00 

348.10 

58.02 

216.00 

52.60 

24.4% 

6.66 

MD011F 

20.10 

1638.75 

81.53 

703.50 

255.50 

36.3% 

6.42 

C005B 

3.70 

206.54 

55.87 

133.20 

32.40 

24.0% 

6.60 

C017A 

23.3 

939.20 

40.32 

419.40 

147.50 

35.2% 

6.38 

Total/Avg 

125.2 

10211.34 

81.56 

46.96 

12.59 

26.8% 

6.48 

Table  7  Sample  Simulation  Output 


The  statistics  give  necessary  into  how  the  available  aircraft  can  be  used.  Another  statistic 
tracked  is  the  potential  total  cost  if  all  scheduled  and  available  CRAF  aircraft  are  utilized  during 
the  simulated  month.  The  cost  of  the  simulated  schedule  is  calculated  with  the  cost  savings  as 
the  difference  between  the  number  of  CRAF  actually  used  and  the  number  of  CRAF  available. 
The  total  cost  includes  a  cancellation  fee  of  $  100,000.  For  validation  purposes,  and  to  determine 
if  the  simulation  is  beneficial,  cost  savings  are  calculated  by  comparing  the  simulation  to  the 
actual  schedule  used  (Table  14). 

Port  hold  times  are  also  calculated.  Port  hold  time  is  the  amount  of  time  a  pallet  spends 

in  port  until  it  is  shipped.  Large  port  hold  times  represent  inefficiencies  in  the  airlift  schedule 

possibly  due  to  not  enough  aircraft  traveling,  or  (as  will  be  shown  in  Chapter  4)  poor  scheduling 

of  airlift.  Due  to  the  nature  of  the  military’s  business,  most  cargo  traveling  to  an  AOR  is 

important  in  some  way  and  military  personnel  cannot  afford  to  wait  long  periods  of  time  for  their 
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shipments.  Although  the  focus  of  this  research  is  not  port  hold  times,  it  is  important  to  monitor 
such  important  factors  to  detennine  positive  or  negative  impacts  based  on  the  changes  this 
research  suggests.  The  simulation  calculates  port  hold  times  for  each  individual  pallet  as  the 
difference  between  the  date  a  pallet  is  shipped  and  the  date  of  pallet  arrival  at  the  APOE. 

Model  Verification  &  Validation 

The  model  is  verified  in  a  step-by-step  fashion.  The  Monte  Carlo  simulation  is  verified 
by  reviewing  randomization,  the  distribution,  and  the  values  of  the  generated  pallet  weights,  the 
arrival  dates,  and  the  locations.  Verification  also  involves  stepping  through  the  cargo  loading 
algorithm  to  determine  accurate  loop  entry,  day  selection,  location  selection  and  available 
aircraft  detennination.  If  aircraft  are  available,  it  is  observed  if  pallets  proceed  to  be  assigned  to 
the  aircraft  until  the  minimum  or  maximum  constraints  are  met.  All  types  of  aircraft  must  be 
approached  for  availability  if  pallets  still  remain  to  be  delivered.  Lastly,  internal  run  statistics 
are  reviewed  to  determine  if  the  pallets  are  being  assigned  to  aircraft,  if  all  aircraft  can  be  filled, 
if  the  aircraft  constraints  are  met,  if  all  pallets  are  shipping  when  possible,  and  that  the  algorithm 
is  moving  across  all  three  phases  and  is  properly  tenninating. 

Validation  of  the  simulation  involved  comparison  to  past  records  and  data  splitting. 

T  ACCS  AS  provides  the  option  to  run  both  a  simulation  with  random  pallet  data  and  a  simulation 
using  the  actual  pallet  data  for  the  specified  month  as  pulled  from  the  GATES  records.  The 
random  pallet  data  uses  the  actual  pallet  number  and  pallet  weight  distributions  for  the  specified 
month.  Both  simulations  use  the  actual  airlift  schedule  to  ensure  that  the  necessary  parameters 
are  properly  compared.  This  allows  the  user  to  view  the  simulated  pallet  statistics  side-by-side  in 
matching  charts.  This  cross  check  validates  that  the  model  is  perfonning  as  necessary  and  that 
any  statistical  calculations  are  accurate.  It  also  allows  the  user  to  immediately  determine 
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whether  or  not  the  actual  schedule  is  optimal  or  where  improvements  can  be  made  in  the  future 
by  comparing  the  number  of  pallets  shipped,  the  tons  shipped,  and  the  number  of  CRAF 
cancelled. 


Real  Pal 

et  Statistics 

#  missions 

Tonnage 

Tons/msn 

pit  offer 

pallets  used 

%  pallets  utilized 

Avg  pallet  wt 

B74710 

12 

796.37 

66.36 

504.00 

438.90 

87.1% 

1.82 

B74720 

33.2 

3597.19 

108.35 

2224.40 

1944.30 

87.4% 

1.85 

B74740 

6.50 

785.42 

120.92 

481.00 

400.00 

83.1% 

1.97 

C005A 

6.00 

344.77 

57.46 

216.00 

188.00 

87.0% 

1.83 

MD011F 

14.10 

962.28 

68.24 

493.50 

476.20 

96.5% 

2.02 

C005B 

1.80 

95.17 

52.82 

64.80 

56.60 

87.2% 

1.70 

C017A 

5.4 

202.00 

37.48 

97.20 

89.70 

92.2% 

2.26 

Total/Avg 

79 

6783.21 

85.86 

51.66 

45.49 

88.1% 

1.89 

Table  8  Example  of  Simulation  Results  with  Actual  Pallet  Data 


Summary 

There  are  three  very  distinct  components  to  the  problem  methodology.  The  first  portion 
required  sorting  and  parsing  the  original  GATES  data  set  for  FY  2005-2011.  Excel  macros  are 
developed  to  perform  these  actions  and  to  separate  the  information  necessary  for  a  month-by¬ 
month  examination.  The  reduced  data  is  compiled  with  a  single  monthly  cargo  demand  time 
series  used  to  develop  an  appropriate  forecasting  model.  A  number  of  suitable  models  are 
constructed  that  provided  accurate  forecast  data.  After  observing  each  model  in  different 
forecasting  scenarios,  the  set  was  cut  down  to  the  five  best  models.  These  models  are  further 
evaluated  in  Chapter  4.  The  final  component  was  to  develop  a  simulation  of  a  monthly  pallet 
schedule  and  individual  pallet  weights  to  be  assigned  to  a  predetermined  aircraft  schedule.  The 
simulation  includes  a  cargo  loading  algorithm  that  assigns  available  pallets  to  available  aircraft 
based  on  day  of  the  month  and  APOD. 
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IV.  Analysis 


Model  Analysis 

It  is  difficult  to  accurately  provide  long  term  forecasts;  a  simple  trend  may  increase  or 
decrease  deviations  through  time.  After  finding  the  best  five  models,  an  iterative  comparison 
determined  which  model  could  most  accurately  predict  cargo  tonnage  for  multiple  scenarios.  It 
is  important  to  repeat  the  comparison  process  while  adding  in  new  data  to  see  how  the  models 
accommodate  the  change  and  adjust  their  forecasts.  For  example,  one  of  the  models  had  a  very 
low  forecast  initially  for  one  month  out,  but  after  including  the  data  that  accounted  for  the  drastic 
increase,  the  model  corrected  and  began  to  forecast  better.  The  question  is  if  these  models  are 
only  good  in  an  instance  or  if  forecasting  (and  these  models  specifically)  can  be  regularly  applied 
to  cargo  demand.  This  process  proved  that  these  models  are  capable  of  accommodating  change 
and  continually  predicting  cargo  tonnage  on  a  yearly,  quarterly,  or  monthly  basis. 

The  intervention  model,  seasonal  exponential  smoothing,  ARIMA,  IMA,  and  SARIMA 
models  are  all  repeatedly  updated  by  adding  new  observations  to  the  data  set.  All  of  the 
predicted  values  are  compared  to  the  actual  values  and  two  models  stood  out:  the  intervention 
model  and  the  SARIMA  model.  Although  the  intervention  model  seemed  the  best  model,  the 
lack  of  real  BOG  numbers  caused  the  forecast  to  change  when  provided  different  real  world 
values.  These  two  models  are  better  at  making  significant  adjustments  to  rapid  increases  and 
decreases.  This  allows  for  relatively  accurate  predictions.  Forecasting  is  by  no  means  an  exact 
science,  but  the  ability  to  adjust  to  change  is  important.  The  seasonal  exponential  smoothing 
performed  the  worst,  due  to  its  inability  to  quickly  respond  to  significant  changes. 

Table  9  provides  a  brief  look  into  the  final  model  analysis  and  selection.  Each  of  the  five 

models  was  rerun  12  times,  each  time  adding  in  a  new  data  point.  The  top  row  is  the  number  of 
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observations  that  are  included  in  the  model  development  and  the  second  row  is  the  number  of 
months  forecasted.  The  percentage  deviations  indicate  how  well  the  models  forecasting  abilities 
adapt  beginning  with  a  full  year  forecast  all  the  way  down  to  forecasting  the  final  month  of  the 
year.  The  “Avg”  column  on  the  far  left  is  the  average  percent  deviation  of  the  forecasted  values. 
The  intervention  model  regularly  performs  better  than  all  the  other  models.  The  ARIMA(  1,1,1) 
and  the  IMA  models  are  the  next  best  models  and  display  more  improvement  and  greater 
accuracy  than  the  last  two  models. 


69  obs 

70  obs 

71  obs 

72  obs 

73  obs 

74  obs 

75  obs 

76  obs 

77  obs 

78  obs 

79  obs 

80  obs 

12M 

11M 

10M 

9M 

8M 

7M 

6M 

5M 

4M 

3M 

2M 

1M 

Avg 

Inter 

IS.  71% 

10.25% 

10.29% 

11.06% 

11.98% 

13.21% 

13.57% 

12.51% 

15.17% 

20.88% 

6.23% 

12.01% 

12.57% 

SES 

14.47% 

16.83% 

14.86% 

16.89% 

22.22% 

24.37% 

16.54% 

14.14% 

16.64% 

23.24% 

15.31% 

29.39% 

18.74% 

ARIMA 

11.37% 

12.94% 

10.85% 

10.91% 

13.96% 

15.05% 

13.98% 

12.34% 

14.95% 

23.70% 

7.44% 

14.02% 

13.46% 

IMA 

11.37% 

19.87% 

12.30% 

11.29% 

17.73% 

18.56% 

14.95% 

11.82% 

14.42% 

24.47% 

9.40% 

16.50% 

15.22% 

SARIMA 

21.75% 

18.04% 

12.45% 

13.61% 

17.60% 

19.59% 

16.20% 

12.58% 

15.08% 

20.80% 

7.23% 

13.95% 

15.74% 

Table  9  Percentage  Deviation  Iterative  Model  Comparison 


Table  10  compares  the  one  month  ahead  forecast  accuracies.  The  most  appropriate 
solution  methodology  for  the  problem  requires  not  only  a  reasonable  forecast  for  a  full  year,  but 
accurate  predictions  for  the  upcoming  month  in  order  to  finalize  airlift  schedules.  Due  to  the 
nature  of  using  real  world  data,  there  will  always  be  instances  where  data  points  are  drastically 
different  from  the  previous  points.  This  is  the  case  with  the  cargo  data  since  it  depends  on 
current  operations  in  the  AOR.  Despite  that,  many  of  the  forecasts  are  within  10%  of  the  actual 
value  and  in  some  cases  within  5%  or  less.  Based  on  average  month-to-month  accuracy,  the 
intervention  model,  ARIMA,  and  the  IMA  models  are  the  best.  The  IMA  model  has  a  lower 
average  than  the  SARIMA,  but  the  SARIMA  has  closer  minimum  and  maximum  deviations. 
The  SARIMA  will  be  used  in  the  simulation  along  with  the  intervention  model  and  the  ARIMA 
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model.  Using  the  long-term  forecast  and  the  short  term  forecast,  the  ARIMA  model  provides 
better  extended  forecast  values  than  simple  exponential  smoothing. 


Inter 

SES 

ARIMA 

IMA 

SARIMA 

69obs 

23.45% 

10.15% 

6.42% 

6.42% 

10.16% 

70obs 

3.86% 

20.73% 

12.88% 

18.89% 

19.66% 

71  obs 

1.82% 

13.22% 

7.56% 

13.02% 

13.04% 

72  obs 

8.77% 

11.96% 

11.37% 

12.95% 

11.57% 

73  obs 

1.76% 

1.42% 

1.46% 

2.12% 

0.67% 

74  obs 

5.37% 

22.99% 

6.40% 

9.05% 

12.84% 

75  obs 

13.80% 

20.41% 

14.31% 

16.42% 

20.96% 

76  obs 

1.63% 

7.07% 

2.52% 

2.51% 

4.20% 

77  obs 

14.69% 

17.18% 

15.71% 

17.43% 

17.75% 

78  obs 

37.97% 

35.27% 

40.05% 

38.61% 

35.18% 

79  obs 

1.24% 

2.55% 

2.00% 

5.16% 

0.99% 

80  obs 

12.01% 

29.39% 

14.02% 

16.50% 

13.95% 

Avg  10.53%  16.03%  11.22%  13.26%  13.42% 
Max  F 31.91%  35.27%  '  40.05%  ' 38.61%  ' 35.18% 

Min  r  1.24%  r  1.42%  r  1.46%  "  2.12%  0.67% 


Table  10  One  Month  Ahead  Forecast  Accuracy  and  Average 


If  forecasting  is  included  in  the  normal  planning  procedures,  the  best  scenario  is  to  rerun 
all  of  the  models  each  forecasting  attempt  and  to  compare  the  goodness  of  fit  criteria,  focusing 
on  the  MSE,  and  using  whichever  model  is  the  most  accurate.  Unfortunately,  the  same  model 
may  not  necessarily  be  applied  each  time;  however,  repeated  analysis  shows  that  the  seasonal 
exponential  smoothing  model  is  consistently  the  top  model  of  those  examined.  Given  the 
validation,  seasonal  exponential  smoothing  was  not  be  the  first  model  selected,  but  the  analyses 
show  it  to  be  a  very  good  model  still.  The  other  models  all  rotated  positions  since  each  model 
has  strengths  and  weaknesses  in  their  prediction  abilities. 


Real  Pallet  Scenario 

Analysis  was  conducted  by  developing  and  running  scenarios  using  TACCSAS.  Initially, 
it  is  most  important  to  run  the  system  and  observe  how  similar  the  results  are  when  using  the  real 
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pallet  data  with  the  real  airlift  schedule  and  the  cargo  loading  algorithm  compared  to  the  actual 
statistics  collected  from  that  month’s  GATES  data.  This  provides  further  system  validation  and 
shows  any  initial  process  improvement  by  using  the  cargo  algorithm  versus  the  current  loading 
methods.  Figure  18  and  Figure  19  show  the  number  of  stons  of  cargo  and  pallets  shipped  by 
using  real  data  within  the  simulation  and  by  simulating  the  pallet  data  compared  to  the  actual 
shipping  values.  The  red  (actual),  green  (real  data  simulation),  and  the  purple  (Monte  Carlo 
simulation)  lines  all  run  very  close  together.  It  also  appears,  by  the  position  of  the  blue  line  (total 
cargo  or  pallets  available),  that  each  month  has  residual  cargo.  Residual  cargo  must  either  be 
factored  into  the  next  month’s  forecast  for  any  additional  aircraft  purchases  or  placed  on  enroute 
channel  missions.  The  simulation  shows  that  the  planners  appear  to  load  the  aircraft  well  each 
month. 


Tons  Shipped 


Pallets  Shipped 


Figure  18  Actual  Schedule  Simulation  (Cargo) 


Figure  19  Actual  Schedule  Simulation  (Pallets) 


Readers  may  notice  the  large  deviation  of  the  real  data  simulation  from  the  actual  values 
on  the  data  point  “Feb-1 1 .”  This  occurs  because  the  simulation  uses  a  higher  minimum  capacity 
of  75%  on  the  amount  of  tonnage  or  the  number  of  pallets  on  the  aircraft  before  departure  is 
allowed.  The  constraint  is  put  into  place  due  to  current  plans  allowing  planes  to  depart  once  they 
are  50%  full  (Figure  1);  and  actual  data  shows  that  planes  depart  with  even  lower  utilization 
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rates.  Increasing  utilization  rates  reduces  the  number  of  aircraft  necessary.  For  the  mentioned 
data  point,  there  were  many  days  at  the  end  of  the  month  where  these  constraints  were  not  met 
and  airlift  not  scheduled,  causing  pallets  to  be  left  in  inventory  at  the  end  of  the  simulation.  If 
aircraft  can  take  off  with  lower  capacities,  then  more  partially  full  flights  are  scheduled.  The 
problem  is  detennining  the  best  percentage  capacity  to  use. 

The  tonnage  and  the  pallet  values  that  are  used  as  the  maximum  carrying  capacities  are 
planning  values.  The  aircraft  can  carry  more  and  in  many  cases,  the  additional  ability  is  used. 
The  tonnage  utilization  from  a  full  year  of  actual  data  is  between  40%  and  125%.  The  aircraft 
are  not  capable  of  carrying  more  pallets  than  the  set  number  of  pallet  positions  built  into  an 
airframe. 

After  comparing  the  actual  statistics  and  the  simulation  results,  we  can  refer  to  the 
original  schedule  and  determine  where  to  make  improvements.  For  “Oct- 10,”  1,594  pallets 
needed  to  be  shipped  totaling  3,279.44  tons.  The  remaining  pallets  are  either  rolled  over  to  the 
next  month  or  transferred  on  enroute  missions.  Nearly  all  of  the  pallets  can  be  airlifted  with  the 
“extra”  CRAF.  This  means  that  the  additional  CRAF  can  be  used  if  placed  elsewhere  in  the 
schedule.  When  comparing  the  actual  schedule  to  the  schedule  the  simulation  produces,  on  the 
few  days  where  more  than  two  CRAF  are  scheduled  (instances  of  3,  4,  and  5  CRAF  on  a  single 
day),  the  additional  CRAF  are  usually  cancelled.  There  are  flights  scheduled  for  every  day  in  the 
actual  schedule,  and  it  would  be  an  easy  adjustment  to  ensure  that  there  is  an  even  distribution  of 
departures  across  the  month.  This  would  lead  to  less  cancellations  and  actually  using  the  aircraft 
that  are  purchased. 


48 


Redistributed  Scenario 


To  analyze  this  conjecture,  a  simulation  is  run  with  a  schedule  consisting  of  the 
same  aircraft,  with  the  same  APOD,  scheduled  on  different  days.  Figure  20  and  Figure  21  show 
that  there  are  no  significant  differences  if  the  same  flights  are  rescheduled.  However,  Figure  22 
shows  that  a  modified  schedule  allows  the  same  pallets  to  ship  the  same  month  while 
consistently  using  less  aircraft  than  the  original  schedule.  After  reviewing  this  study  it  is 
apparent  that  cancellations  are  easy  to  generate,  but  the  important  consideration  is  how  to  better 
schedule  the  flights  to  better  accomplish  the  mission. 
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Figure  20  Redistributed  Simulation  (Cargo)  Figure  21  Redistributed  Simulation  (Pallets) 


Missions  Flown 


Figure  22  Redistributed  Simulation  (Missions  Flown) 


Times  Series  Model  Scenarios 

The  forecasted  weight  and  the  cargo  loading  algorithm  can  be  used  to  simulate  what  the 
potential  schedule  and  airlift  capabilities  would  look  like  for  the  upcoming  months.  As  more 
data  is  included  into  the  forecasting  model,  in  most  cases,  a  better  fitting  model  is  developed  and 
the  predictions  improve.  In  repeated  testing,  some  model  instances  show  greater  than  99% 
accuracy.  The  simulation  shows  planners  how  many  aircraft  they  need  to  move  a  predicted 
number  of  pallets  and  a  predicted  number  of  tons. 

The  predicted  pallet  numbers  are  derived  from  the  forecasted  cargo  weight.  The  mean  of 
a  previous  month’s  pallet  weights  is  found  and  the  forecasted  tons  are  divided  by  the  mean  pallet 
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weight  to  get  the  forecasted  number  of  pallets.  The  pallet  weight  is  then  simulated  as  a  random 
value  using  the  nonnal  distribution.  The  average  weight  of  the  simulated  pallets  is  then  very 
close  to  the  actual  pallet  weight,  providing  additional  validation  to  the  simulation  construction. 
Even  if  the  forecasted  value  is  lower,  planners  are  still  aware  of  how  all  of  the  predicted  pallets 
are  moved  in  the  simulation.  For  example  if  it  took  about  33  aircraft  to  airlift  over  3,000  tons, 
then  it  would  it  would  be  fair  to  say  they  would  need  an  additional  16  aircraft  to  move  an  extra 
1,500  tons. 

The  forecasted  tonnage  from  the  three  previously  identified  models  are  used  in  the 
simulation  to  simulate  how  the  potential  pallet  numbers  and  weights  can  be  distributed  amongst  a 
predetennined  airlift  schedule  with  specified  aircraft  already  assigned  to  days  of  the  month.  The 
simulation  is  replicated  and  the  run  results  averaged.  Figure  23  shows  the  forecasted  cargo 
demand  from  the  three  models  in  relation  to  the  actual  cargo  shipped  during  FY 1 1 .  The 
forecasts  appear  to  run  across  the  minimum,  maximum,  and  average  actual  values.  Since  the 
forecasts  are  so  close  to  the  actual  shipping  values,  the  results  of  the  simulation  show  very 
similar  charts  across  amount  of  cargo  shipped  and  the  number  of  pallets  shipped.  Figure  24  is  a 
comparison  of  how  many  missions  are  flown  in  each  month.  It  is  expected  that  the  two  lower 
forecasts  will  require  fewer  missions,  but  the  higher  forecast  (Intervention  Model)  also  flies 
fewer  missions  to  ship  a  slightly  greater  number  of  cargo  pallets. 
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Forecast  Comparison 


Missions  Flown 


Figure  23  Forecasted  Cargo  Demand 


Figure  24  Forecast  Simulation  (Mission  Flown) 


Table  1 1  shows  the  minimum  number  of  missions  flown  by  each  type  of  aircraft 
identified  to  transport  the  total  tonnage  of  3,241.86  stons  as  well  as  the  average  number  of  pallets 
flown  on  each  type  of  aircraft  for  “Oct-1 1.”  An  important  value  in  Table  1 1  is  the  total  number 
of  aircraft  necessary  to  accommodate  the  forecasted  number  of  tons  or  pallets.  In  this  case,  the 
intervention  model  forecasts  that  only  52  aircraft  total  or  36  CRAF  are  needed.  When  validated 
against  the  actual  GATES  data,  the  intervention  model  forecasted  higher  than  the  actual  tonnage, 
but  the  simulation  shows  that  the  current  schedule  is  capable  of  airlifting  the  actual  number  of 
pallets  (1,594  pallets)  and  the  full  tonnage  (3,279.44  tons)  while  still  cancelling  13  CRAF. 
Although  the  forecast  is  higher  than  the  actual  value,  CRAF  can  be  cancelled  and  42%  of  the 
cost  saved. 
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#  missions 

Tons/Msn 

Tons  Utilization 

Pallets/Msn 

Pallet 

Utilization 

B74720 

21.43 

70.11 

62.60% 

32.74 

99.20% 

B74740 

3.30 

68.13 

54.95% 

32.23 

97.65% 

MD011F 

8.39 

70.69 

83.16% 

34.50 

98.57% 

B74710 

2.71 

67.61 

86.68% 

30.44 

92.26% 

C005M 

1.00 

60.27 

98.80% 

27.87 

77.43% 

C005B 

5.15 

59.65 

97.78% 

28.42 

78.95% 

C017A 

9.47 

39.11 

86.91% 

18.52 

102.89% 

Total 

Total 

Average 

Total 

Average 

Total/Avg 

51.46 

3241.86 

75.43% 

1529.70 

96.86% 

Avg  Pallet  Hold  Time  2.77days 

Table  11  Intervention  Model  Simulation  -  Pallet  Statistics  “Oct-11” 


Full  Schedule  Scenario 

A  simulation  provides  a  great  deal  of  flexibility  in  its  modeling  capabilities.  It  may  be 
useful  for  planners  to  know  how  many  aircraft  are  necessary  or  how  many  additional  aircraft  are 
needed  to  airlift  every  pallet  if  enroute  missions  are  no  longer  available.  The  original  schedules 
show  that  not  enough  aircraft  are  scheduled  for  flights  to  OAKN,  leaving  pallets  in  port.  The 
schedules  also  show  that  CRAF  are  primarily  scheduled  for  the  first  three  weeks  and  not  evenly 
scheduled  through  the  end  of  the  month.  Table  12  shows  the  modified  daily  airlift  schedule  for 
each  month. 
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Aircraft 

Type 

APOD 

C017A 

Organic 

OAKN 

C005A 

Organic 

OAKN 

C005B 

Organic 

OAKN 

B74710 

CRAF 

OAIX 

B 74720 

CRAF 

OAIX 

B 74740 

CRAF 

OAIX 

MD011F 

CRAF 

OAIX 

B74710 

CRAF 

OAKB 

B 74720 

CRAF 

OAKB 

B 74740 

CRAF 

OAKB 

MD011F 

CRAF 

OAKB 

Table  12  Full  Daily  Schedule 


Figure  25  and  Figure  26  show  that  it  is  possible  to  move  every  pallet  available.  February 
is  still  the  outlier  due  to  the  capacity  constraint.  Figure  27  shows  that  a  small  increase  in  the 
number  of  monthly  missions  is  enough  to  accommodate  the  pallet  inventory.  It  is  equal  to 
approximately  an  additional  15  flights  per  month  or  8  CRAF. 
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Tons  Shipped 

Pallets  Shipped 

A - A.  * 

- - - * - ^ 

s 

♦  Number  of  Tons  Available 

B  Actual  Tons  Shipped 

♦  Number  of  Pallets  Available 

-1-  Actual  Pallets  Shipped 

A  Full  Schedule  Sim 

A  Full  Schedule  Sim 

Oct-10  Nov-10  Dec-10  Jan-11  Feb-11  Mar-11  Apr-11  May-11  Jun-11  Jul-11  Aug-11  Sep-11 

Oct-10  Nov-10  Dec-10  Jan-11  Feb-11  Mar-11  Apr-11  May-11  Jun-11  Jul-11  Aug-11  Sep-11 

Figure  25  Full  Simulation  (Cargo)  Figure  26  Full  Simulation  (Pallets) 


Figure  27  Full  Simulation  (Missions  Flown) 


All  Organic  Scenario 

Planners  can  attempt  to  replicate  all  of  their  airlift  efforts  with  just  organic  aircraft.  If 
there  is  ever  a  situation  where  APODs  are  too  hostile  for  CRAF  aircraft,  it  might  be  useful  to 
know  how  many  missions  organic  aircraft  would  have  to  fly  in  order  to  fill  the  gaps.  To  run  this 
scenario,  each  CRAF  flight  on  the  schedule  is  replaced  with  a  similarly  sized  organic  aircraft. 
Although  C-5s  are  capable  of  carrying  a  great  deal  of  cargo,  the  planning  parameters  for  stons 
carried  are  quite  low  causing  them  to  reach  their  maximum  stons  capacity  before  reaching  their 
pallet  capacity.  Figure  30  shows  that  the  same  number  of  missions  are  used  to  move  less  cargo 
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as  shown  in  Figure  28  and  Figure  29.  More  organic  aircraft  would  be  required  to  perfonn  the 
same  amount  of  work,  which  may  not  be  feasible  due  to  the  size  of  the  organic  aircraft  inventory. 


Figure  30  Organic  Simulation  (Missions  Flown) 


Results 

Table  13  is  a  summary  table  of  the  scenario  airlift  mission  number  results  compared  to 
the  actual  data.  With  exception  to  the  Full  Schedule  and  Organic  scenarios,  each  simulation 
scenario  uses  less  CRAF  aircraft  to  airlift  the  same  number  of  pallets  or  cargo  stons. 
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Actual 

Full 

Real  Pallet 

Monte  Carlo 

Redistributed 

Organic 

CRAF 

B74710 

14 

484 

14.207 

581.77 

10 

444.49 

7.0936 

448.21 

6.9211 

443.55 

0 

0 

B74720 

292 

351.44 

268.52 

272.23 

274.21 

0 

B74730 

3 

0 

3 

3.1 

3 

0 

B74740 

80 

174 

73.468 

74.16 

68.475 

0 

MD011F 

95 

42.118 

89.5 

91.624 

90.95 

0 

Organic 

C005A 

21 

272 

9.9 

355.93 

14 

216.09 

17.249 

240.1 

13.976 

220.68 

272 

665.64 

C005B 

60 

69.102 

46.745 

50.049 

44.857 

156.01 

C005M 

14 

0 

11.229 

13.142 

12.451 

83.081 

C017A 

176 

276.93 

143.12 

158.66 

148.39 

153.54 

KC010A 

1 

0 

1 

1 

1 

1 

Total  756  937.7  660.58  688.31  664.23  665.64 


Table  13  Scenario  Summary  -  Total  Airlift  Missions 


Table  15  is  a  summary  of  the  monthly  cost  of  schedules  associated  with  each  scenario. 
The  number  of  CRAF  aircraft  necessary  for  each  scenario  is  shown  along  with  the  number  of 
cancellations  possible.  Cost  savings  is  calculated  as  previously  discussed  to  include  the  sunk 
cost  to  schedule.  If  the  number  of  CRAF  flights  scheduled  is  less  than  the  actual  number, 
cancellations  are  calculated.  Simulation  of  each  of  the  scenarios  shows  improvements  can  be 
made  in  nearly  every  month  to  achieve  a  more  cost  effective  schedule.  The  redistributed 
schedule  shows  the  most  number  of  cancellations  and  therefore  the  greatest  savings. 
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Actual  GATES 

Full  Schedule  Sim 

Real  Pallet  Sim 

Monte  Carlo  Sim 

Redistributed 

Oct- 10 

CRAF  Flights 

49 

40 

30 

29 

34 

Cancellations 

8 

18 

19 

15 

Cost  of  Schedule 

$  22,540,000.00 

$  19,295,431.75 

$  15,646,000.00 

$  15,398,316.67 

$  17,058,550.79 

Nov-10 

CRAF  Flights 

41 

44 

32 

33 

33 

Cancellations 

0 

9 

8 

8 

Cost  of  Schedule 

$  18,860,000.00 

$  20,240,000.00 

$  15,620,000.00 

$  15,980,000.00 

$  15,755,019.84 

Dec- 10 

CRAF  Flights 

42 

55 

41 

41 

40 

Cancellations 

0 

0 

0 

2 

Cost  of  Schedule 

$  19,320,000.00 

$  25,354,360.32 

$  18,884,022.22 

$  18,961,747.62 

$  18,372,774.60 

Jan-11 

CRAF  Flights 

36 

51 

36 

36 

36 

Cancellations 

0 

0 

0 

0 

Cost  of  Schedule 

$  16,560,000.00 

$  23,460,000.00 

$  16,560,000.00 

$  16,560,000.00 

$  16,780,197.62 

Feb-11 

CRAF  Flights 

43 

49 

42 

44 

40 

Cancellations 

0 

1 

0 

3 

Cost  of  Schedule 

$  19,780,000.00 

$  22,540,000.00 

$  19,296,511.90 

$  20,420,769.05 

$  18,501,579.37 

Mar-11 

CRAF  Flights 

43 

49 

41 

42 

41 

Cancellations 

0 

1 

1 

1 

Cost  of  Schedule 

$  19,780,000.00 

$  22,562,032.54 

$  19,119,065.08 

$  19,229,227.78 

$  19,097,707.94 

Apr-11 

CRAF  Flights 

34 

45 

36 

35 

36 

Cancellations 

0 

0 

0 

0 

Cost  of  Schedule 

$  15,640,000.00 

$  20,484,073.81 

$  16,468,638.89 

$  16,120,207.14 

$  16,464,659.52 

May-11 

CRAF  Flights 

37 

48 

33 

33 

35 

Cancellations 

0 

3 

4 

2 

Cost  of  Schedule 

$  17,020,000.00 

$  22,013,099.21 

$  15,691,107.14 

$  15,395,561.90 

$  16,125,163.49 

Jun-11 

CRAF  Flights 

45 

51 

43 

45 

41 

Cancellations 

0 

1 

0 

4 

Cost  of  Schedule 

$  20,700,000.00 

$  23,460,000.00 

$  19,908,896.03 

$  20,789,170.63 

$  19,235,010.32 

Jul-11 

CRAF  Flights 

33 

42 

33 

33 

32 

Cancellations 

0 

0 

0 

1 

Cost  of  Schedule 

$  15,180,000.00 

$  19,324,600.00 

$  15,053,773.81 

$  15,084,440.48 

$  14,820,000.00 

Aug-11 

CRAF  Flights 

35 

48 

34 

34 

35 

Cancellations 

0 

1 

0 

0 

Cost  of  Schedule 

$  16,100,000.00 

$  22,080,000.00 

$  15,740,000.00 

$  15,786,159.52 

$  16,100,000.00 

Sep-11 

CRAF  Flights 

46 

54 

43 

43 

42 

Cancellations 

0 

2 

3 

3 

Cost  of  Schedule 

$  21,160,000.00 

$  24,840,000.00 

$  20,045,860.32 

$  19,995,411.11 

$  19,624,600.00 

Total  Cancellations 

8 

36 

35 

39 

Total  Cost 

$  222,640,000.00 

$  265,653,597.62 

$  208,033,875.40 

$  209,721,011.90 

$  207,935,263.49 

Total  Savings 

$  (43,013,597.62) 

$  14,606,124.60 

$  12,918,988.10 

$  14,704,736.51 

Table  14  Scenario  Comparison  of  Cost  Savings 


Summary 

Continuous  one-step  ahead  forecasting  with  the  five  models  selected  shows  there  are 
various  models  capable  of  providing  regular  accurate  forecasts.  Having  an  actual  forecasting 
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ability  is  substantial  in  tenns  of  reserving  CRAF.  Rather  than  providing  weak  forecasts  and 
under-committing  on  the  “fixed  buy,”  planners  can  provide  much  greater  commitment  and 
incentive  to  the  CRAF  participants  and  the  short  term  buy.  Planners  may  reduce  the  number  of 
CRAF  they  over-purchase  by  applying  the  forecast  to  the  simulation.  The  simulation  easily 
displays  how  to  move  the  forecasted  pallets  with  the  least  number  of  aircraft  possible,  allowing 
schedulers  to  examine  any  shortcomings.  With  just  a  few  examples,  it  is  easy  to  show  that  the 
simulation  can  provide  different  options  by  changing  a  few  simple  parameters.  The  take  away 
from  the  simulation  is  that  there  is  money  to  be  saved  in  the  current  scheduling  whether  it  is  a 
matter  of  reducing  the  number  of  CRAF  perfonning  these  channel  missions  or  reducing  the 
number  of  CRAF  flying  enroute  missions.  It  is  feasible  to  conduct  a  full  organic  schedule, 
especially  if  wartime  ACL  parameters  are  being  used,  but  not  necessarily  preferred. 
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V.  Conclusion 


This  research  builds  upon  the  foundation  for  a  potential  forecasting  tool  and  a  cargo 
loading  simulation.  Time  series  forecasting  provides  an  accurate  method  for  predicting  cargo 
demand  and  while  a  “best”  model  may  not  be  consistent  with  every  new  forecast  generated,  the 
forecasts  are  still  quite  accurate.  The  goal  of  continued  forecasting  research  is  to  determine  if 
advanced  time  series  models  can  be  applied  to  generate  more  accurate  forecasts.  The 
development  of  new  models  leads  to  the  additional  question  of  whether  or  not  forecasts  and 
simulation  can  lead  to  accurate  airlift  schedules  with  the  objective  to  maximize  airlifted  cargo 
tons.  The  suggested  models,  the  capabilities,  the  potential  impacts,  and  areas  for  improvement  or 
future  investigation  are  presented  below. 

Results 

This  research  provides  a  brief  overview  of  the  current  airlift  results  and  continues  with  a 
detailed  examination  of  how  and  where  improvements  can  be  made.  Two  past  research  projects 
provide  a  strong  foundation  for  thesis  (DeYoung,  2012;  Lindstrom,  2012).  This  research  focuses 
on  incorporating  mathematical  models  into  a  long  standing  planning  process,  how  to  apply  the 
models  for  reliable  results,  and  how  to  generate  an  increasingly  accurate  outlook  from  12  months 
out  to  1  month  out.  The  techniques  utilized  in  this  process,  time  series  forecasting,  Monte  Carlo 
simulation,  and  heuristic  optimization,  each  offer  individual  benefits  towards  increasing  value 
and  efficiency. 

Forecasting  models  are  constructed  having  high  R  values  and  strong  goodness-of-fit 
criteria  to  create  accurate  forecasts  for  short  tenn  and  long-tenn  cargo  demand.  The  models  are 
validated  by  using  data  splitting.  The  MSEs  are  compared  amongst  multiple  models  through 
monthly  forecast  iterations  for  a  full  year  to  observe  how  the  models  handled  new  data  and 
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reacted  to  the  changes.  This  research  demonstrates  that  forecasting  can  be  a  very  useful  tool  and 
can  easily  fit  a  cargo  data  set.  There  are  a  variety  of  models  applied  to  the  data  set  with  good 
results,  and  each  can  be  updated  regularly  as  new  data  are  received  to  improve  model  predictive 
capabilities. 

The  Monte  Carlo  simulation  produces  very  good  estimates  of  cargo  pallet  weight  by 
using  simple  distributions  and  good  samples  of  pallet  arrivals  and  destinations  with  discrete 
probability  distributions.  The  cargo  loading  algorithm  effectively  and  efficiently  loads  aircraft 
with  pallets,  paired  by  destinations,  and  shows  cost  savings  and  areas  for  modification.  The 
algorithm  is  validated  through  data  splitting  and  the  results  showed  the  same  cargo  movement 
perfonnance  can  be  achieved  with  far  less  aircraft  and  large  savings.  The  simulation  is  easily 
used  to  generate  potential  schedules  and  show  related  statistics  allowing  planners  to  make 
changes  or  improvements  or  to  plan  ahead. 

Past  research  emphasized  the  need  for  forecasting  airlift  sustainment  cargo  demand  for 
the  AORs  (DeYoung,  2012)  which  is  why  Afghanistan  is  the  focus.  While  this  is  the  main  basis 
of  the  research,  simulations  show  there  is  room  for  improvement  in  the  overall  scheduling  of 
airlift,  the  loading  of  aircraft,  and  the  understanding  of  cancellation  fees  and  port  hold  time.  If 
these  issues  exist  outside  of  planning  for  the  AOR,  then  it  would  be  beneficial  to  explore 
applications  to  other  aerial  ports  as  well. 

Research  Conclusion 

The  model  forecast  charts  in  Appendix  C  show  that  with  time,  forecasts  can  sometimes 
become  less  accurate.  A  use  of  these  charts  is  whether  or  not  the  forecasts  are  an  improvement 
over  the  current  moving  average  model  used.  The  improvement  shown  provides  a  direct  benefit 
to  the  DoD’s  incentive  relationship  with  its  CRAF  partners.  A  more  accurate  projection  of 
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CRAF  business  one  year  out  can  confidently  increase  the  size  of  the  “fixed  buy”  and  reduce  the 
size  of  “expansion  buys”  The  methods  that  are  used  in  this  research  are  easily  applied  and  the 
goodness-of-fit  is  directly  dependent  on  how  planners  choose  to  filter  the  input  data. 

This  research  found  no  past  research  that  applies  simulation  to  such  a  broad  planning 
effort.  Most  simulation  models  or  products  appear  to  be  applied  to  the  physical  loading  of  the 
aircraft  or  for  cargo  routing/deliveries.  The  benefit  of  simulation  is  that  it  can  be  tailored  to  the 
need  and  its  performance  can  be  validated  with  real  data  sets.  The  forecasts  and  the  simulation 
focus  on  improved  planning  overall  and  show  potential  financial  impacts  or  savings  from  the 
airlift  schedule  used.  The  focus  of  the  simulation  is  improving  the  planning  process  to  reduce 
costs.  This  includes  the  cost  benefit  of  cancelling  previously  scheduled  missions  as  well.  The 
main  conclusion  derived  from  analyzing  data  from  FY201 1  is  that  scheduling  of  airlift  needs 
improvement  and  depending  on  how  the  application  of  these  methods  continue,  there  are 
potential  cost  savings  in  reducing  the  number  of  CRAF  flying  each  month  and  cancelling 
unnecessary  CRAF. 

A  direct  result  of  the  research  is  a  modeling  tool  that  incorporates  the  techniques  from 
Chapters  3  and  4.  This  Excel  model  forecasts  cargo  demand  using  multiple  models,  uses  the 
forecasted  demand  to  simulate  a  schedule  of  aircraft  for  that  month  based  on  an  input  schedule  or 
a  sample  schedule,  and  provides  results  of  the  simulation  showing  areas  of  improvement  and 
potential  cost  impacts  or  savings.  Users  can  include  aircraft  of  their  choosing,  schedule  aircraft 
as  they  please,  use  multiple  statistical  distributions  to  simulate  pallet  weights  and  review  how 
each  individual  aircraft  is  loaded.  This  tool  can  improve  USTRANSCOMs  current  planning 
capabilities. 
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Recommendations 


USTRANSCOM  planners  should  begin  to  use  forecasting  techniques  in  their  airlift 
planning.  Multiple  models  can  be  applied  repeatedly  on  a  monthly  basis  while  injecting  new 
data  and  comparing  it  with  actual  cargo  data.  It  is  also  recommended  that  weekly  data  be  used 
and  tested  for  a  six  week  forecast  to  incorporate  last  minute  changes.  Multiple  models  should  be 
applied  due  to  the  nature  of  the  model  types  and  the  real  data  set,  but  this  research  shows  that 
using  these  models  is  beneficial.  Side-by-side  comparisons  for  a  time  period  provide  planners  an 
evaluation  period  as  well  as  a  learning  period.  Assuming  TACCSAS  performs  well,  it  is  an 
adaptive  system  ready  to  incorporate  new  data  and  simulate  real  schedules  and  efficiencies  based 
off  of  the  forecasted  demand.  It  is  recommended  that  TACCSAS  be  used  as  soon  as  possible  to 
evaluate  current  schedules  and  for  future  planning. 

Future  Research 

Due  to  the  difficulty  in  retrieving  BOG  data  as  well  as  the  observation  that  BOG 
dependent  forecasting  models  did  not  provide  improved  forecasts,  future  models  should  be 
developed  using  just  cargo  demand.  Using  the  one  time  series  will  also  simplify  future  model 
development.  In  addition  to  using  a  single  input,  past  research  (DeYoung,  2012)  shows  that 
weekly  demand  values  can  be  used  to  generate  accurate  forecasts  as  well.  Actual  application 
may  require  a  combination  of  both  monthly  and  weekly  forecasts  in  order  to  enhance  current 
planning  efforts. 

The  simulation  can  easily  be  applied  to  other  APODs  and  other  APOEs  to  detennine  if 
modifications  can  be  made  to  their  airlift  scheduling.  For  example,  all  Dover  AFB  data  can  be 
replaced  with  that  from  Travis  AFB  to  detennine  if  savings  are  available  in  PACAF  missions  or 
if  schedules  can  be  performed  in  a  manner  to  provide  better  coverage  and  more  airlift. 
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While  there  are  a  number  of  advanced  forecasting  techniques,  artificial  neural  networks 
(ANN)  in  particular  are  quite  accurate  and  are  a  black  box  system  that  can  “learn”  nearly  any 
data  set.  It  may  be  worth  the  time  and  the  effort  to  apply  ANNs  to  the  time  series  data  to 
potentially  derive  a  more  accurate  forecast.  Additional  software  or  add-ins  are  necessary  to 
perform  the  computations,  but  many  of  these  solutions  are  readily  available  and  fairly  straight 
forward  in  their  use  (with  some  understanding  of  the  topic).  A  brief  experimentation  is 
performed  with  the  same  set  of  time  series  cargo  data  and  time-delay  neural  networks  to  observe 
the  performance.  Recurrent  neural  networks  are  also  applied  to  the  data  series.  The  steps  and 
the  results  are  found  in  Appendix  F. 
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Appendices 


Appendix  A 

The  time  series  of  the  monthly  total  of  the  cargo  weights  can  be  reviewed  in  the  table 
below.  Cargo  numbers  are  pulled  from  the  GATES  database  and  manually  sorted  and  calculated. 
Boots  of  the  Ground  values  are  taken  from  (Belasco,  2011).  For  the  model  development,  the  last 
estimated  value  of  98,000  BOG  was  used  for  FY20 1 1 . 


Date 

Tons  of  Cargo 

BOG 

Date 

Tons  of  Cargo 

BOG 

Jan-05 

626.24 

18,700 

Oct-08 

2,083.65 

33,000 

Feb-05 

355.21 

20,300 

Nov-08 

1,796.79 

33,000 

Mar-05 

1098.62 

20,900 

Dec-08 

1,848.90 

32,500 

Apr-05 

1321.85 

19,500 

Jan-09 

1,709.50 

32,800 

May-05 

1868.07 

20,000 

Feb-09 

2,395.76 

35,900 

Jun-05 

1765.68 

19,200 

Mar- 09 

2,814.97 

38,350 

Jul-05 

1564.97 

21,100 

Apr-09 

2,710.91 

39,000 

Aug-05 

1464.06 

17,400 

May-09 

2,397.15 

44,700 

Sep-05 

1239.89 

18,000 

Jun-09 

2,784.60 

55,100 

Oct- 05 

888.69 

17,800 

Jul-09 

3,240.28 

56,500 

Nov-05 

1141.64 

17,400 

Aug-09 

3,761.82 

62,600 

Dec-05 

965.61 

18,500 

Sep-09 

1,684.92 

62,300 

Jan-06 

968.55 

20,300 

Oct- 09 

3,129.91 

65,800 

Feb-06 

1317.77 

22,700 

Nov-09 

3,240.86 

67,500 

Mar-06 

1171.79 

20,000 

Dec-09 

4,015.49 

69,000 

Apr-06 

1246.90 

23,300 

Jan-10 

4,669.42 

70,200 

May-06 

1123.86 

21,800 

Feb- 10 

3,820.45 

74,600 

Jun-06 

1430.04 

22,300 

Mar- 10 

4,551.66 

85,000 

Jul-06 

1166.78 

20,800 

Apr- 10 

5,605.41 

87,600 

Aug-06 

1013.93 

19,700 

May- 10 

5,152.41 

89,700 

Sep-06 

970.40 

20,400 

Jun-10 

4,962.08 

91,775 

Oct- 06 

1340.10 

19,800 

Jul-10 

4,393.06 

95,925 

Nov-06 

972.08 

20,500 

Aug- 10 

3,762.09 

95,920 

Dec-06 

664.56 

21,800 

Sep-10 

2,974.05 

98,000 

Jan-07 

701.91 

26,000 

Oct- 10 

3,300.60 

Feb-07 

728.49 

24,800 

Nov- 10 

4,149.92 

Mar-07 

2897.11 

24,400 

Dec- 10 

4,536.60 

Apr-07 

1742.49 

23,900 

Jan-11 

5,069.22 

May-07 

1111.17 

26,400 

Feb-11 

4,852.68 

Jun-07 

900.47 

23,800 

Mar- 11 

4,529.13 

Jul-07 

1135.81 

24,000 

Apr-11 

4,037.11 

Aug-07 

1072.69 

24,000 

May-11 

4,370.84 

Sep-07 

873.20 

24,500 

Jun-11 

5,318.32 

Oct- 07 

840.92 

24,400 

Jul-ll 

3,691.94 

Nov-07 

550.83 

24,800 

Aug-11 

4,014.77 

Dec-07 

744.61 

24,600 

Sep-11 

4,941.08 

Jan-08 

756.80 

27,000 

Feb-08 

981.68 

28,000 

Mar-08 

1166.90 

28,800 

Apr-08 

920.68 

33,100 

May-08 

1028.56 

35,600 

Jun-08 

1539.14 

34,000 

Jul-08 

1469.30 

33,700 

Aug-08 

1623.82 

34,200 

Sep-08 

1611.04 

33,500 
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Appendix  B 


Monthly  Afghanistan  Cargo  Demand  Models 

The  following  graphs  show  the  monthly  historical  cargo  time  series  (black  dots)  with  the 
fitted  forecasting  model  (red  line).  A  95%  confidence  interval  is  provided  (blue  lines)  around  the 
forecasted  value. 


Figure  31  Monthly  Forecast  Model  Graphs 
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Appendix  C 


Top  5  Models  Iterative  Comparison 

An  iterative  comparison  performed  with  the  five  best  models  is  shown  in  the  figures 
below.  An  initial  12  month  forecast  was  produced  by  each  model  and  the  overall  accuracies  are 
detennined.  A  new  data  point  was  then  included  into  the  data  set  and  new  forecasts  are 
generated  for  the  remaining  months.  The  purpose  of  this  is  to  find  out  if  these  models  are  only 
good  in  an  instance  or  if  forecasting  (and  these  models  specifically)  can  be  regularly  applied  to 
this  type  of  data  set.  This  process  proved  that  these  models  are  capable  of  accommodating 
change  and  continually  predicting  cargo  tonnage  on  a  yearly,  quarterly,  or  monthly  basis. 
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Table  15  All  Model  Residual  and  Percentage  Deviations 

69obs  70obs  71obs  72obs  73obs  74obs  75obs  76obs  77obs  78obs  79obs  80obs 


12  Month  11  Month  10  Month  9  Month  8  Month  7  Month  6  Month  5  Month  4  Month  3  Month  2  Month  1  Month 


Intervention  Model 

Month  70 

-774.01 

Month  71 

-390.09 

160.14 

Month  72 

-326.19 

158.30 

82.72 

Month  73 

-13.21 

498.51 

490.46 

444.50 

Month  74 

-76.91 

186.67 

214.35 

196.75 

-85.54 

Month  75 

-511.58 

-184.08 

-137.62 

-141.49 

-291.32 

-243.26 

Month  76 

-1195.10 

-699.47 

-643.18 

-640.41 

-729.19 

-709.98 

-557.09 

Month  77 

-671.52 

-377.32 

-315.92 

-309.94 

-370.57 

-364.63 

-285.80 

71.44 

Month  78 

335.37 

564.43 

628.47 

636.01 

588.35 

588.20 

633.36 

822.11 

781.51 

Month  79 

-1171.06 

-1064.78 

-999.37 

-991.08 

-1032.76 

-1035.71 

-1005.86 

-893.69 

-909.55 

-1401.93 

Month  80 

-719.24 

-743.36 

-677.25 

-668.59 

-707.51 

-711.75 

-688.86 

-611.50 

-616.04 

-847.68 

-49.90 

Month  81 

517.51 

182.25 

248.72 

257.56 

219.90 

215.08 

234.81 

296.34 

296.98 

175.03 

554.62 

593.26 

Seasonal  Exponential  Smoothing 

Month  70 

-335.09 

Month  71 

629.62 

860.42 

Month  72 

916.64 

1147.44 

599.62 

Month  73 

1316.37 

1547.54 

999.18 

606.05 

Month  74 

1068.84 

1300.01 

751.65 

358.52 

-68.77 

Month  75 

48.62 

279.79 

-268.57 

-661.70 

-1088.99 

-1041.17 

Month  76 

-424.46 

-193.30 

-741.65 

-1134.78 

-1562.08 

-1514.26 

-823.88 

Month  77 

86.19 

317.36 

-231.00 

-624.13 

-1051.43 

-1003.61 

-313.23 

308.84 

Month  78 

917.24 

1148.41 

600.05 

206.92 

-220.37 

-172.56 

517.82 

1139.90 

913.70 

Month  79 

-657.31 

-426.15 

-974.50 

-1367.63 

-1794.93 

-1747.11 

-1056.73 

-434.66 

-660.86 

-1302.06 

Month  80 

-281.05 

-49.88 

-598.24 

-991.37 

-1418.67 

-1370.85 

-680.47 

-58.40 

-284.60 

-925.80 

-102.54 

Month  81 

1208.38 

1439.55 

891.19 

498.06 

70.77 

118.59 

808.97 

1431.04 

1204.84 

563.64 

1386.89 

1452.17 

rH 

rH 

rH 

< 

cc 

< 

Month  70 

-211.79 

Month  71 

370.11 

534.57 

Month  72 

606.06 

740.86 

342.76 

Month  73 

1043.30 

1160.74 

829.52 

576.18 

Month  74 

757.64 

865.54 

566.17 

356.90 

-70.70 

Month  75 

377.42 

480.50 

195.33 

7.22 

-342.51 

-289.95 

Month  76 

-165.34 

-64.36 

-344.18 

-522.73 

-832.50 

-789.59 

-577.65 

Month  77 

120.44 

220.87 

-58.08 

-232.92 

-523.02 

-485.14 

-313.23 

110.35 

Month  78 

1021.31 

1122.05 

841.73 

667.62 

386.34 

421.69 

573.47 

916.83 

835.59 

Month  79 

-651.05 

-549.53 

-832.35 

-1007.21 

-1285.48 

-1251.31 

-1109.24 

-806.88 

-872.74 

-1478.45 

Month  80 

-373.90 

-271.34 

-557.24 

-733.61 

-1011.97 

-978.24 

-840.43 

-558.26 

-616.14 

-1103.81 

-80.33 

Month  81 

506.86 

610.61 

321.35 

143.07 

-137.04 

-103.33 

33.08 

306.10 

252.22 

-175.91 

636.24 

692.50 

rH 

rH 

< 

Month  70 

-211.79 

Month  71 

370.11 

783.80 

Month  72 

606.06 

1131.05 

590.67 

Month  73 

1043.30 

1624.24 

1076.29 

656.26 

Month  74 

757.64 

1368.27 

812.75 

386.77 

-102.95 

Month  75 

377.42 

1005.29 

442.19 

10.27 

-486.27 

-409.78 

Month  76 

-165.34 

473.84 

-96.82 

-534.68 

-1038.04 

-960.50 

-662.71 

Month  77 

120.44 

768.14 

189.90 

-253.90 

-764.07 

-685.49 

-383.70 

109.72 

Month  78 

1021.31 

1676.19 

1090.38 

640.63 

123.65 

203.28 

509.07 

1008.99 

927.04 

Month  79 

-651.05 

10.39 

-583.00 

-1038.69 

-1562.49 

-1481.81 

-1172.02 

-665.61 

-748.61 

-1425.30 

Month  80 

-373.90 

293.79 

-307.18 

-768.80 

-1299.42 

-1217.69 

-903.90 

-390.99 

-475.06 

-1160.45 

-207.32 

Month  81 

506.86 

1180.66 

572.13 

104.56 

-432.88 

-350.10 

-32.31 

487.09 

401.97 

-292.13 

673.36 

815.47 

SARI  MA(0,l,l)x(  1,0, 1)12 

Month  70 

-335.39 

Month  71 

59.24 

815.86 

Month  72 

596.49 

1138.31 

591.76 

Month  73 

1272.19 

1596.19 

1046.59 

586.53 

Month  74 

1191.59 

1343.61 

789.54 

363.36 

-32.72 

Month  75 

997.21 

684.94 

137.97 

-229.82 

-605.16 

-581.62 

Month  76 

627.94 

179.32 

-373.00 

-831.38 

-1271.75 

-1250.24 

-846.32 

Month  77 

1078.29 

570.42 

10.00 

-416.92 

-847.76 

-824.93 

-435.59 

183.49 

Month  78 

2136.59 

1443.48 

880.07 

461.39 

41.55 

65.15 

472.94 

1079.98 

944.21 

Month  79 

615.51 

-181.73 

-751.06 

-1174.03 

-1585.71 

-1561.07 

-1144.70 

-556.59 

-694.22 

-1298.98 

Month  80 

1038.39 

142.99 

-432.29 

-845.83 

-1240.50 

-1214.46 

-786.79 

-229.38 

-368.83 

-939.44 

-39.64 

75 


All  Model  Residuals 
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12  Month  11  Month  10  Month  9  Month  8  Month  7  Month  6  Month  5  Month  4  Month  3  Month  2  Month  1  Month 


Intervention  Model 

Month  70 

23.45% 

Month  71 

9.40% 

3.86% 

Month  72 

7.19% 

3.49% 

1.82% 

Month  73 

0.26% 

9.83% 

9.68% 

8.77% 

Month  74 

1.58% 

3.85% 

4.42% 

4.05% 

1.76% 

Month  75 

11.30% 

4.06% 

3.04% 

3.12% 

6.43% 

5.37% 

Month  76 

29.60% 

17.33% 

15.93% 

15.86% 

18.06% 

17.59% 

13.80% 

Month  77 

15.36% 

8.63% 

7.23% 

7.09% 

8.48% 

8.34% 

6.54% 

1.63% 

Month  78 

6.31% 

10.61% 

11.82% 

11.96% 

11.06% 

11.06% 

11.91% 

15.46% 

14.69% 

Month  79 

31.72% 

28.84% 

27.07% 

26.84% 

27.97% 

28.05% 

27.24% 

24.21% 

24.64% 

37.97% 

Month  80 

17.91% 

18.52% 

16.87% 

16.65% 

17.62% 

17.73% 

17.16% 

15.23% 

15.34% 

21.11% 

1.24% 

Month  81 

10.47% 

3.69% 

5.03% 

5.21% 

4.45% 

4.35% 

4.75% 

6.00% 

6.01% 

3.54% 

11.22% 

12.01% 

Seasonal  Exponential  Smoothing 

Month  70 

10.15% 

Month  71 

15.17% 

20.73% 

Month  72 

20.21% 

25.29% 

13.22% 

Month  73 

25.97% 

30.53% 

19.71% 

11.96% 

Month  74 

22.03% 

26.79% 

15.49% 

7.39% 

1.42% 

Month  75 

1.07% 

6.18% 

5.93% 

14.61% 

24.04% 

22.99% 

Month  76 

10.51% 

4.79% 

18.37% 

28.11% 

38.69% 

37.51% 

20.41% 

Month  77 

1.97% 

7.26% 

5.29% 

14.28% 

24.06% 

22.96% 

7.17% 

7.07% 

Month  78 

17.25% 

21.59% 

11.28% 

3.89% 

4.14% 

3.24% 

9.74% 

21.43% 

17.18% 

Month  79 

17.80% 

11.54% 

26.40% 

37.04% 

48.62% 

47.32% 

28.62% 

11.77% 

17.90% 

35.27% 

Month  80 

7.00% 

1.24% 

14.90% 

24.69% 

35.34% 

34.15% 

16.95% 

1.45% 

7.09% 

23.06% 

2.55% 

Month  81 

24.46% 

29.13% 

18.04% 

10.08% 

1.43% 

2.40% 

16.37% 

28.96% 

24.38% 

11.41% 

28.07% 

29.39% 
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Month  70 

6.42% 

Month  71 

8.92% 

12.88% 

Month  72 

13.36% 

16.33% 

7.56% 

Month  73 

20.58% 

22.90% 

16.36% 

11.37% 

Month  74 

15.61% 

17.84% 

11.67% 

7.35% 

1.46% 

Month  75 

8.33% 

10.61% 

4.31% 

0.16% 

7.56% 

6.40% 

Month  76 

4.10% 

1.59% 

8.53% 

12.95% 

20.62% 

19.56% 

14.31% 

Month  77 

2.76% 

5.05% 

1.33% 

5.33% 

11.97% 

11.10% 

7.17% 

2.52% 

Month  78 

19.20% 

21.10% 

15.83% 

12.55% 

7.26% 

7.93% 

10.78% 

17.24% 

15.71% 

Month  79 

17.63% 

14.88% 

22.55% 

27.28% 

34.82% 

33.89% 

30.04% 

21.86% 

23.64% 

40.05% 

Month  80 

9.31% 

6.76% 

13.88% 

18.27% 

25.21% 

24.37% 

20.93% 

13.91% 

15.35% 

27.49% 

2.00% 

Month  81 

10.26% 

12.36% 

6.50% 

2.90% 

2.77% 

2.09% 

0.67% 

6.20% 

5.10% 

3.56% 

12.88% 

14.02% 

IMA(1,1) 

Month  70 

6.42% 

Month  71 

8.92% 

18.89% 

Month  72 

13.36% 

24.93% 

13.02% 

Month  73 

20.58% 

32.04% 

21.23% 

12.95% 

Month  74 

15.61% 

28.20% 

16.75% 

7.97% 

2.12% 

Month  75 

8.33% 

22.20% 

9.76% 

0.23% 

10.74% 

9.05% 

Month  76 

4.10% 

11.74% 

2.40% 

13.24% 

25.71% 

23.79% 

16.42% 

Month  77 

2.76% 

17.57% 

4.34% 

5.81% 

17.48% 

15.68% 

8.78% 

2.51% 

Month  78 

19.20% 

31.52% 

20.50% 

12.05% 

2.32% 

3.82% 

9.57% 

18.97% 

17.43% 

Month  79 

17.63% 

0.28% 

15.79% 

28.13% 

42.32% 

40.14% 

31.75% 

18.03% 

20.28% 

38.61% 

Month  80 

9.31% 

7.32% 

7.65% 

19.15% 

32.37% 

30.33% 

22.51% 

9.74% 

11.83% 

28.90% 

5.16% 

Month  81 

10.26% 

23.89% 

11.58% 

2.12% 

8.76% 

7.09% 

0.65% 

9.86% 

8.14% 

5.91% 

13.63% 

16.50% 

CM 

H 

H 

O 

T— 1 

X 

x— ^ 
x— ^ 

O 

< 

cc 

< 

to 

Month  70 

10.16% 

Month  71 

1.43% 

19.66% 

Month  72 

13.15% 

25.09% 

13.04% 

Month  73 

25.10% 

31.49% 

20.65% 

11.57% 

Month  74 

24.56% 

27.69% 

16.27% 

7.49% 

0.67% 

Month  75 

22.02% 

15.12% 

3.05% 

5.07% 

13.36% 

12.84% 

Month  76 

15.55% 

4.44% 

9.24% 

20.59% 

31.50% 

30.97% 

20.96% 

Month  77 

24.67% 

13.05% 

0.23% 

9.54% 

19.40% 

18.87% 

9.97% 

4.20% 

Month  78 

40.17% 

27.14% 

16.55% 

8.68% 

0.78% 

1.23% 

8.89% 

20.31% 

17.75% 

Month  79 

16.67% 

4.92% 

20.34% 

31.80% 

42.95% 

42.28% 

31.01% 

15.08% 

18.80% 

35.18% 

Month  80 

25.86% 

3.56% 

10.77% 

21.07% 

30.90% 

30.25% 

19.60% 

5.71% 

9.19% 

23.40% 

0.99% 

All  Model  Percentage  Deviations 


76 


Appendix  D 


Monte  Carlo  Simulation  Construction 

Chapter  3  discussed  the  development  of  the  parameters  for  the  Monte  Carlo  Simulation. 
Below  is  a  small  portion  of  the  airlift  schedule  from  October  FY201 1.  The  table  below  shows 
which  aircraft  flew  to  what  destination  on  what  day.  This  schedule  can  be  used  to  assign  pallets 
to. 


Table  16  Monte  Carlo  Simulation  Construction 


Aircraft 

Type 

Destination 

Direct/Enroute 

Day 

B74720 

CRAF 

OAIX 

Direct 

1 

B74720 

CRAF 

OAIX 

Direct 

2 

B74740 

CRAF 

OAIX 

Direct 

3 

B74720 

CRAF 

OAIX 

Direct 

3 

B74720 

CRAF 

OAIX 

Direct 

3 

B74720 

CRAF 

OAIX 

Direct 

4 

B74720 

CRAF 

OAIX 

Direct 

4 

MD011F 

CRAF 

OAIX 

Direct 

4 

B74720 

CRAF 

OAKB 

Direct 

5 

B74720 

CRAF 

OAKB 

Direct 

5 

B74720 

CRAF 

OAIX 

Direct 

5 

MD011F 

CRAF 

OAIX 

Direct 

5 

B74720 

CRAF 

OAIX 

Direct 

6 

B74740 

CRAF 

OAIX 

Direct 

7 

B74720 

CRAF 

OAIX 

Direct 

7 

MD011F 

CRAF 

OAIX 

Direct 

8 

B74720 

CRAF 

OAIX 

Direct 

9 

B74720 

CRAF 

OAIX 

Direct 

9 

MD011F 

CRAF 

OAIX 

Direct 

10 

B74710 

CRAF 

OAKB 

Direct 

10 

MD011F 

CRAF 

OAIX 

Direct 

10 

B74720 

CRAF 

OAIX 

Direct 

10 

B74710 

CRAF 

OAIX 

Direct 

10 

MD011F 

CRAF 

OAIX 

Direct 

11 

MD011F 

CRAF 

OAIX 

Direct 

11 

B74710 

CRAF 

OAKB 

Direct 

12 

B74720 

CRAF 

OAIX 

Direct 

12 

B74720 

CRAF 

OAIX 

Direct 

13 

77 


The  right  hand  table  shows  a  small  portion  of  the  pallets  that  arrive  to  Dover  AFB,  with 
their  destination  and  their  arrival  date.  The  right  hand  table  shows  how  each  pallets’  arrival  date 
to  Dover  AFB  is  counted,  the  probability  of  occurrence  is  found,  and  then  a  discrete  probability 
distribution  is  fonned. 


PLT GROSS WT 

APODJCAO 

Arrival  Date 

DayCount 

Day 

Probability 

CumProb 

0.59 

OAKN 

1 

72 

1 

4.52% 

0.00% 

5.01 

OAIX 

1 

52 

2 

3.26% 

4.52% 

2.19 

OAIX 

1 

26 

3 

1.63% 

7.78% 

32 

4 

2.01% 

9.41% 

2.94 

OAIX 

1 

70 

5 

4.39% 

11.42% 

1.51 

OAKN 

1 

75 

6 

4.71% 

15.81% 

1.40 

OAKN 

1 

89 

7 

5.58% 

20.51% 

4.89 

OAIX 

1 

79 

8 

4.96% 

26.10% 

2.50 

OAIX 

1 

43 

9 

2.70% 

31.05% 

2.35 

OAKN 

1 

28 

10 

1.76% 

33.75% 

1.97 

OAIX 

1 

12 

11 

0.75% 

35.51% 

1.09 

OAIX 

1 

48 

12 

3.01% 

36.26% 

1.94 

OAIX 

1 

53 

13 

3.32% 

39.27% 

2.35 

OAKB 

1 

57 

14 

3.58% 

42.60% 

92 

15 

5.77% 

46. 17% 

1.64 

OAIX 

1 

44 

16 

2.76% 

51.94% 

2.58 

OAIX 

1 

37 

17 

2.32% 

54.71% 

2.57 

OAIX 

1 

34 

18 

2.13% 

57.03% 

1.16 

OAKN 

1 

53 

19 

3.32% 

59.16% 

2.84 

OAIX 

1 

45 

20 

2.82% 

62.48% 

2.58 

OAIX 

1 

59 

21 

3.70% 

65.31% 

2.23 

OAIX 

1 

47 

22 

2.95% 

69.01% 

1.86 

OAIX 

1 

43 

23 

2.70% 

71.96% 

4.31 

OAIX 

1 

14 

24 

0.88% 

74.65% 

39 

25 

2.45% 

75.53% 

2.46 

OAIX 

1 

66 

26 

4.14% 

77.98% 

2.55 

OAKN 

1 

59 

27 

3.70% 

82.12% 

1.83 

OAIX 

1 

48 

28 

3.01% 

85.82% 

2.16 

OAIX 

1 

68 

29 

4.27% 

88.83% 

2.44 

OAIX 

1 

49 

30 

3.07% 

93.10% 

4.45 

OAKN 

1 

61 

31 

3.83% 

96. 17% 

The  table  below  shows  how  each  pallets’  destination  is  counted,  the  probability  of 
occurrence  is  found,  and  then  a  discrete  probability  distribution  is  formed. 


Destination  Direct/Enroute  Location# 

Count  Probability  CumProb 

OAIX 

Direct 

1 

1103 

69.20% 

0.00% 

OAKB 

Direct 

2 

153 

9.60% 

69.20% 

OAKN 

Direct 

3 

338 

21.20% 

78.80% 
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Appendix  E 


Decision  Support  System  -  Strategic  Airlift  Simulation  Description 

A  decision  support  system  is  a  computer  based  information  system  that  can  be  used  for 
management,  operations,  or  for  planning  purposes.  It  uses  data  for  processing  or  for  analytical 
purposes  to  aid  the  decision  maker  in  his  or  her  decision-making  process.  A  DSS  was 
constructed  with  a  graphical  user  interface  (GUI)  so  that  the  sponsor  at  USTRANSCOM  could 
perfonn  the  same  actions  as  discussed  throughout  my  research,  but  without  assistance.  The  DSS 
requires  a  working  knowledge  of  the  CRAF  scheduling  program  and  a  walk-through  of  the  DSS. 
Screenshots  and  brief  instructions  will  be  laid  out  below  as  a  brief  tutorial  for  potential  users. 


Please  Enable  Macros  by  clicking  the  options  button  (shown  in 
picture  below)  and  selecting  enable  macros. 


Home  Insert  Page  Layout  Formulas  Data  Review 


*  u  Callbri  -  11  -  A'  a'  *  m  m  V/- 

S  Format  Painter  B  1  U  A’  99  9  i  M 


^  Security  Warning  Macros  have  been  disabled. 

Options... 

P3  -  f* 

A  B  C  0 

E  I 

F  G 

1 


Click  Start  Program  when  finished. 


Initial  screen  when  opening  TACCSAS 
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Welcome 


The  TACC  Strategic  Airlift  Simulation  (SAS)  allows  the  user  to  simulate  transporting  cargo 
pallets.  Simulation  options  include  a  full  simulation  with  all  estimated  parameters  ,  a 
simulation  based  off  of  a  previous  airlift  schedule,  a  simulation  using  real  pallet  data,  and 
simulation  using  forecasted  cargo  tonnage  for  one  month,  three  month,  or  a  full  year.  SAS 
simulates  all  pallets  traveling  from  Dover  AFB  to  Afghanistan  using  CRAF  and  Organic 
military  aircraft.  The  user  must  simulation  parameters  to  include  number  of  runs  and  cargo 
pallet  weight  distribution  parameters  and  the  type  of  schedule  to  be  run.  The  user  will  be 
able  to  add  and  delete  aircraft  available  for  a  particular  schedule,  change  aircraft  capability' 
parameters,  and  add  or  delete  scheduled  aircraft  departures.  Once  the  simulation  has  run, 
the  user  will  be  able  to  view  and  compare  the  collected  statistics. 


Click  the  Start  Button  to  begin 


Start 


Exit  Program 


Welcome  screen  when  starting  TACCSAS 


Menu 


Run  Simulation 

View  Cargo  Data 

Edit  Aircraft 

Edit  Calendar 

View  Simulated  Pallet  Statistics 

View  Real  Pallet  Data  Statistics 

View  Summary  Statistics 

Exit  DSS 

Menu  appears  when  selecting  “Start” 
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Select  “Run  Simulation.”  This  screen  provide  seven  options  for  simulation.  The 
simulation  choices  include  the  “fixed  buy”  and  the  “expansion  buy”  forecast  options.  The 
recommended  number  of  simulation  iterations  if  100.  Upon  opening,  defaults  will  be  provided 
within  the  selection. 
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If  the  user  selects  a  forecast  simulation,  forecast  parameters  will  appear  allowing  for  a 
variety  of  schedules  to  used  in  the  simulation. 
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Aircraft  Parameters 


Add  Aircraft  to  List 


Please  set  the  schedule  values 
Enter  Aircraft  Name 
Select  Aircraft  Type 
Enter  Aircraft  Tons 
Enter  Number  of  Pallets 


IT 


Please  enter  minimum  values 
as  a  percentage  in  decimal 
format  (50%  =  .  5) 


Enter  Minimum  Aircraft  Tons 
Enter  Minimum  Number  of  Pallets 

Add 


Change  Aircraft  Parameters 


Please  set  the  schedule  values 
Select  Aircraft 
Select  Aircraft  Type 
Change  Aircraft  Tons 
Change  Number  of  Pallets 


3 

- 


Please  enter  minimum  values 
as  a  percentage  in  decimal 
format  (50%  =  .5) 


Change  Minimum  Aircraft  Tons 


Change  Minimum  #  of  Pallets 


Save 


Delete  Aircraft  From  List 

Please  set  the  schedule  values 

Pick  T ype  of  Aircraft 


“3 

Delete 


Return 


Select  “Edit  Aircraft”  from  Menu.  The  user  is  allowed  to  change  the  ACL  parameters  for 
each  type  of  aircraft.  Users  can  also  add  new  CRAF  or  Organic  aircraft  with  capacity  levels  as 
well 
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Add  Or  Delete  Dates  •  ls^3®l 


Select  “Edit  Calendar”  from  Menu.  The  user  can  choose  a  month  and  add  CRAF  or 
Organic  flights  to  the  schedule  or  remove  them. 


Run  # 

Avg  Port  Hold  Of 

Avg  Port  Hold 
For  Pallets  Not 

Delivered 

Pallets  Not 

Weight  Not 

Total  Weight 

Total  Weight 

All  Pallets 

Shippe  d 

Shipped 

Generated 

Shipped 

1 

1.96 

2.49 

1480 

3084.38 

6558.89 

3474.51 

2 

2.06 

2.59 

1554 

3235.98 

6558.89 

3322.91 

3 

1.81 

2.31 

1537 

3200.93 

6558.89 

3357.96 

4 

1.90 

2.38 

1560 

3243.84 

6558.89 

3315.05 

5 

1.96 

2.49 

1480 

3084.38 

6558.89 

3474.51 

6 

1.96 

2.48 

1492 

3111.02 

6558.89 

3447.87 

7 

1.96 

2.49 

1480 

3084.38 

6558.89 

3474.51 

8 

1.69 

2.15 

1505 

3138.44 

6558.89 

3420.45 

9 

1.96 

2.49 

1480 

3084.38 

6558.89 

3474.51 

10 

1.42 

1.81 

1533 

3197.97 

6558.89 

3360.92 

Select  “View  Simulated  Pallet  Statistics”  or  “View  Real  Pallet  Statistics”  from  Menu. 
The  user  can  view  a  large  number  of  statistics  provided  for  each  run  of  the  simulation.  The 
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statistics  are  provided  for  pallets/tons  airlifted  and  per  individual  aircraft  type.  The  next  two 


figures  are  also  included  in  this  selection. 


Run  # 

CRAF 

CRAF 

Cost  Of 

Potential  Total 

Cost  Savings 

Flown 

Cancelled 

Sche  dule 

Cost 

1 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

2 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

3 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

4 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

5 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

6 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

7 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

8 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

9 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

10 

30 

15 

$  15,300,000 

$ 

20,700,000 

$  5,400,000 

Run  # 

Aircraft  ID 

#  of  Missions 

Tons 

Trans  fe  red 

Tons  Per 

Mission 

Tonnage 

Utilization 

Pallets  Per 

Mission 

Pallet 

Utilization 

Potential 

Pallet 

Positions 

Actual 

Pallets 

Shipped 

Total  Pallet 

Utilization 

Avg  Pallet 
Weight 

1 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 

2 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 

3 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 

4 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 

5 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 

6 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 

7 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 

8 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 

9 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 

10 

B 74720 

21 

2882.24 

137.25 

1.23 

72.67 

1.08 

1407.00 

1526.00 

1.08 

1.89 
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Appendix  F 


Contact  Information 

If  anyone  is  interested  in  the  code  that  was  used  in  Excel  to  perfonn  all  of  the  operations  and 

analysis  mentioned  earlier,  please  refer  to  the  contact  infonnation  below. 

Dr.  Jeffery  Weir  Taylor  Leonard,  Capt.  (USAF) 

Jeffery.weir@afit.edu  Taylor.Leonard.l@us.af.mil 
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Problem  Statement 

Current  cargo  demand  forecasts  are  inaccurate  and  result  in 
over/under  purchasing  of  Civil  Reserve  Air  Fleet  (CRAF)  aircraft 
No  technical  method  of  determining  how  many  CRAF  to 
purchase  or  when  to  schedule  them 
CRAF  Cancellations  are  not  included  in  planning  measures 


Objectives 

Examine  effect  of  advanced  mathematical  modeling  on 
operational  planning  and  scheduling  of  CRAF 
Apply  advanced  forecasting  techniques  to  cargo  demand 
Simulate  cargo  demand  to  ensure  that  airlift  is  being  used  to 
the  DOD's  advantage 


Time  Series  Development 

Construct  forecasting  models  using  different  time  series 
methods 

Select  best  models  based  on  goodness  of  fit  criteria  and  data 
splitting 

For  purpose  of  the  study,  data  splitting  used  to  determine  if 
forecasting  models  can  generate  consistently  accurate 
long/short  term  forecasts 
JMP  used  to  develop  all  model  types 

•  11  "good"  models  total 

•  2  Regression 

■  3  Exponential  Smoothing 

•  4ARIMA 

•  2  Transfer  Function 

Goodness-of-Fit  criteria  were  initially  compared,  all  models  are 
similar  and  no  model  is  better  in  every  aspect 


Forecasted  Cargo  Demand:  Oct  2010  -  Sept  2011 


Capt  Taylor  Leonard 
Advisor:  Dr.  Jeffery  Weir 
Reader:  Dr.  Raymond  Hill 

Department  of  Operational  Sciences  (ENS) 
Air  Force  Institute  of  Technology 


Sponsor: 

USTRANSCOM/TCAC 


Forecasting  Model  Inputs 

Cargo  Tonnage  forecast  from  the  best  fitting  model  is  required 
Model  requires  an  estimated  number  of  pallets  for  a  single  month 
period 

Number  of  pallets  determined  by  dividing  the  forecasted  cargo  tonnage 
by  average  pallet  weight  taken  from  historical  data 
Pallet  weights  are  then  simulated  by  normal  distribution  with 
predetermined  mean  and  variance  from  historical  data 


Monte  Carlo  Pallet  Simulation 

Develop  a  Monte  Carlo  simulation  to  represent/replicate  current  airlift 
scenarios  based  on  pallet  parameters 

Construct  heuristic  assignment  model  (Cargo  Loading  Algorithm)  to 
assign  pallets  to  available/scheduled  aircraft 
1  Includes  aircraft  pallet  and  tonnage  maximums  and  minimums 
Run  Monte  Carlo  simulation  using  forecasted  demand  and  run  Cargo 
Loading  Algorithm  to  simulate  optimal  aircraft  needs  and  sample 
schedule 


Cargo  Loading  Algorithm 

For  each  j  =  1  to  Days  in  Month  (a.  Jan = 31  Ays) 

ForeaA/  =  1  tokcstionMmixrfa.  1  =CAKN) 

For  each  k = 1  to  >inher  d  Aircraft  Types  (ex.  B747-200  is  one  type) 

If  email  aircraft = "CRAF"  aid  onrmt  airedt  flies  to;  and  is  available  on  i  Then 

While  Total  tons  on  aircraft  <  MaxTcns  And  Total =Metscn  Aircraft  <  Nfeftllets 
For  each  pallet  not  yet  slipped  by  date:  going  to  location; 

Increment  Total  Tons  and  Total  s  Ml« 

If  Total  tons  on  aircraft  >  MnTans  Or  Total  -  Mets  an  Aircraft  >MnftiIletsThen 
Urn  schedule  loaded  aircraft 
If  aircraft  is  not  leaded  then  cancel 


$  (43.0U  WB] 


S  ?  20-072. 616  67 


i  16,780,19 7.61 


19.3«J10-U 


Conclusions 

•  Advanced  forecasting  models  fit  cargo  data 

•  Multiple  forecasts  with  high  accuracy 

•  Forecasting  ability  for  multi-step  and  single-step 

•  Beneficial  to  DOD  and  CRAF  participants  by  increasing  "fixed 
buy" 

•  Reduce  "expansion  buy" 

•  Simulation  allows  schedulers  to  identify  shortcomings 

•  Can  see  where  CRAF  can  be  moved  to  generate  more  pallet 
movement 

•  Can  see  where  schedule  is  lacking  in  #  flight  or  specific  routes 

•  If  standards  are  in  place  to  fly  a  certain  amount  of  cargo  on 
enroute  missions 

•  Potential  cost  savings  by  reducing  number  of  CRAF  flown  each  month 
cancelling  CRAF  when  not  needed 


Appendix  G 


Vita 


Captain  Taylor  J.  Leonard  graduated  from  Gig  Harbor  High  School  in  Gig  Harbor, 
Washington.  He  entered  the  United  States  Air  Force  Academy  (USAFA)  in  Colorado  Springs, 
Colorado  where  he  graduated  with  a  Bachelor  of  Science  degree  in  Operations  Research  and 
commissioned  in  2008. 

His  first  assignment  was  at  Eglin  AFB  as  a  Command  and  Control  Test  Analyst  in  the 
46th  Test  Squadron  (TS).  In  October  2010,  he  served  as  Deputy  Director  of  the  Datalink  Test 
Facility  at  the  46TS.  In  September  201 1,  he  entered  the  Graduate  School  of  Engineering  and 
Management,  Air  Force  Institute  of  Technology.  Upon  graduation,  he  will  be  assigned  as  a 
faculty  member  of  the  Department  of  Mathematical  Sciences  at  USAFA. 
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